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INTRODUCTION 

In 1961+ Tlie Institute for Mathematical Studies in the Social Sciences at 
Stanford University received a grant from the U.S. Office of Education to 
develop and implement a computer-assisted instruction (CAI) program in initial 
reading and mathematics. The project, known as the Stanford CAI project, is 
a continuation of earlier research hy Patrick Suppes and Richard Atkinson in 
the area of CAI; SUppes is responsible for the development of the mathematics 
curricultim and Atkinson for initial reading. At the beginning of the project 
two major hurdles had to be overcome. There were no lesson materials in either 
mathematics or reading suitable for CAE presentation, and an integrated tutorial 
CAI system had not yet been designed and produced by a single manufacturer. The 
development of the curricula and the development of the system have been carried 
on as parallel efforts over the past three years with each having a decided 

influence on the other. 

This paper is a report of the progress of the Stanford CAI Beading Program 
with particular reference to the school year 1966-6? when the initial classes 
of first-grade students received a major portion of their daily reading instruc- 
tion on a CAI tutorial system. This first year's operations must he consxdered 
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essentially as an extended debugging of both the system and the lesson materials, 
nevertheless, some Interesting comments can be made on the results of this shake- 
dovn experience in regard to both the feasibility of CAt instruction for first- 
grade students and on the intact of such instruction on the overaU learning 

process. 

GENERAL DISCUSSION OF CAL 

Before describing the Stanford Project, a few general remarks about coiiqEruter- 
assisted instruction and CAE systems might be in order. Three levels of con^uter- 
assisted instruction m^ be defined. Discrimination between levels is based not 
on hardware and software considerations but primarily on the cosgilexity and 
sophistication of the student-system interaction. The most advanced student, 
system interaction mt^r be achieved with a single teletype terminal and the most 
primitive Interaction m^ require some hi^ complex programming. 

At the ba^c interaction level are those systems which present a fixed, 
linear sequence of problems. Student errors may be corrected in a variety of 
ways (e.g., a prosit may be given in the form of a partial answer, or the entire 
correct response may be furnished following an error) but no real-time decisions 
are made for introducing unique teaching strategies or instructional materials 
on the basis of a student response. Such sj)stems have been termed drill and 
practice" systems and are exemplified by the Stanford Drill and Practice Program 

in Aritbmetic and in Spelling. 

At the other extreme of the interaction scale are "dialogue" programs of 
the type under Investigation at Bolt, Beranak and Hewman and at Stanford Univer- 
sity. The goal of the "dialogue" approach is to provide the richest possible 
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student- system interaction where the student is free to construct unrestricted 
natural- language responses, ask questions and, in general, exercise aljnost 
complete control over the sequence of learning events. 

"Tutorial" programs lie between the above extremes of student- system inter- 
action. Tutorial programs have the capability for real-time decision-making 
and instructional branching contingent on a single response or on some subset 
of a student's response history. Such programs allow the students to follow 
separate and diverse pathways through the curricul-um based on their individual 
performance patterns. The probability is high in a tutorial program that no 
two students will encounter exactly the same sequence of lesson materials. 

However, student responses are somewhat limited since they must be chosen from 
a prescribed set of responses or constructed in such a manner that relatively 
simple text analysis will be sufficient for their evaluation. The Stanford 
CAI Reading Program is tutorial in nature and it is this level of student- system 
interaction that will be discussed in this paper. 

CAI; A Tool for Teaching . 

Computer-assisted instruction has a theoretical basis, as indeed does all 
programmed learning, in the notion that immediate reinforcement facilitates 
learning. For the human learner, reinforcement may come thix)ugh verbal praise 
acting as a reward or through simple knowledge of the results of one's own actions. 
In the reading program correct responses are rewarded by such verbal messages 
as "good," "you're doing fine," "right," etc. Since studies have shown that 
reinforcement tends to lose its effectiveness when it is continuous and repetitious, 
verbal rewards in the reading program are given on an intermittent basis. Imme- 
diate feedback is provided through reward messages^ through the presentation of 
the next problem and also through "wrong answer" messages. 
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Uninformed criticism of computer-assisted instruction often carries an 
implied component of fear that the instructional process will somehow become 
dehumanized, that the students will become little automatons themselves. As 
it turns out, however, the elimination of the social intercourse aspect of 
learning through CAE is one of its great strengths. The computer is an eternally 
patient teacher. The machine never becomes angry or threatening. Those of us 
who have spent some years teaching in the classroom are well aware of the fact 
that after repeated student errors it is difficult, if not impossible, to 
restrain certain voice or facial cues which indicate our displeasure. The 
messages coming from the machine, however, are completely free of any such 
threat or anger. The "wrong answer" messages recorded in the quiet of the re- 
cording studio can be a continuously neutral "No, this is the right word. 

Touch it." 

We have found that attention of even very young students can be maintained 
at a high level by appropriate pacing of the material. Their attention is also 
increased by the use of partitions between the response terminals. The individual 
student is not distracted by the actions of his classmates, nor is he carried 
along by their responses. Each individual is independently responsible for inter- 
acting with the learning materials. 

The basic rationale, however, for the use of a computer- assisted instructional 
system as a teaching device, resides in the system' c potential for individualizing 
instruction. If there is any one fact which has been thoroughly established in 
sixty years of intensive investigation in education, it is that a wide range of 
individual differences will be found in any classroom on any dimension one wishes 
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to examine. CAI offers us a tool for tailoring our instructional procedures 
to these individual differences. 

The current paper and pencil programs can accommodate individual differences 
in learning time. A bright student who responds rapidly can complete many more 
frames and cover a greater amount of material in a given time interval than can 
the slow student who responds quite deliberately. However, the nearly unlimited 
branching capability inherent in a CAI system not only allows the students to 
proceed independently in terms of speed, but also permits them to follow essen- 
tially different paths through the curriculum. Branching decisions for each 
student in a CAI system may be contingent upon a single. response, his past history 
of responses, response latency, or some combination of these considerations 
weighted by previously acquired psychometric uata. 

CAI; A Tool for Research . 

The usefulness of CAI as a tool for research resides in two factors: l) the 

control of independent vsiriables and 2) the detailed response data which is re- 
corded by the system. 

Educational research conducted outside the laboratory in an actual school 
situation has long been plagued by the impossibility of contrelling many variables 
inherent in classiXDom organization and in presentation of materials by the class- 
room teacher. CAI, in a sense, brings the laboratory into the school. Our own 
CAI laboratory has achieved a degree of control of environment and presentation 
that has been heretofore impossible in a classroom setting. The temperature 
and the lighting in the terminal room are constant. The immediate environment 
of each student's response terminal is precisely the same as every other student's. 
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The chairs and the machines are identical for all students. Every picture seen 
in the projection device, every hit of orthography or other display on the scope, 
apd every audio message vhlch the student hears, have been previously specified 
and can he as standardised or vs.ried as the experimenter desires. This is not 
to say, of course, that all sources of variation are controlled. The CAI 
facility does achieve, hovever, a degree of control equivalent to that of the 
psychologist's learning research laboratory. Many problems in learning theory, 
which have been investigated rigorously only in a latoratory setting, can nov 

be looked at in an on-going school context. 

The second capability of CAI vhich is extremely Important for research is 

the collection of fine-grained response data. For example, in the Stanford- 
Brentwood CAE laboratory, each response that each student mates is record 
the data tapes. Each response record Includes a complete description of the 
response in terms of coordinates taten from the face of the scope or the teys 
depressed on the typewriter. The response is defined as correct or incorrect; 
and If it is incorrect, it is categorized according to the type of error made. 

The response latency is recorded in tenths of a second. The contents of 31 
counters ar.d 32 switches associated with the student's past history of perfor- 
mance are also recorded with each response on the data tape. 

Another Important use of CAI as a research tool is found in the area of 

U , „=wicnlarlv in the area of mathematical learning models, 
mathematical psychology, particularly in xne a 

The quantity and nature of the response data which may be gathered in the CAI 
system allows the mathematical psychologist to test his various models in a 
situation that Is a much closer approximation to actual classroom learning than 
has existed in the past. Typically, such models have been tested through infra- 
organism behavior or through such contrived taste as paired-associate list 
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learning or probability learning. The kind of seipjentlal response data shlch 
ee are gathering In the Brentwood faclUty will be used In the developnent of 
optimization models for learning (Groen and Atkinson (1966)) Atkinson and 

Shiffrin (1967))* 

CA I: A Tcx)l for Curricrilum Evaluation . 

Another feature of CAE in the educational process is in curriculum evaluation. 
Our current methods of evaluatica are extremely gross, relying on standardized 
tests or specially devised tests given on an intermittent schedule. The best 
that can be expected from such evaluation procedures is to be able to conqpare 
the general outcomes of one method or one curriculum approach to some other method 
or approach. Little or nothing can be said about the efficiency of any specific 
section of the curriculum. It is exactly at this detailed level that CM exhibits 
its greatest power for evaluation. The performance data gathered in the CM 
system may be examined at all levels, from the perspective of overall goals or 
from the perspective of the various strategies and approaches adopted in the 
curriculum; we may examine responses to blocks of homogeneous problem types, 
responses to separate problem types and responses to the individual problems 
themselves. At each level we can look at the students' performance records to 
discover if this section or level or item of the curriculum is functioning in 
the manner for which it was designed. 

eescriptioh of system 



Configuration . 

The Staoford-Brentwood laboratory utilizes an IHl 1500 CAE system. The 



1500 system was designed and constructed by I» engineers in close collaboration 
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with Stanford personnel. The student response temdnsls consist of a cathode 
ray tube (CRT), a modified typeurlter keyloard, a light pen, a fllB projection 

device, and a set of earphones vith an attached inlcrophone. 

The CRT is essentially a television screen on vhlch alpha-nunerlc charac- 
ters end a limited set of graphics (i.e., simple line drawings) can he generated 
uMer cos®uter control. The film projector is a l6 mn. rear-view filmstrip 
projector. Still pictures in black and white or color may he displayed under 
colter control. Each filmstrip, in a self-threading cartridge, contains 102h 
frames which may he accessed randomly hy aeans of a binary code along one edge 

of the film. 



The major response device used in the reading program is the li^t pen. 

The light pen is a H^t-sensltlve prohe which registers a portion of the CRT 
trace comprising the CRt raster. The precise interval between the initiation 
of the sweep trace and registry of the trace by the 11^ pen indicates where 
the light pen has been touched to the screen. This location is stored as a 
set of coordinates in computer memory. ISese coordinates are evaluated by the 
systems program and compared against predefined coordinates in the lesson program. 
This cosparison permits evaluation of a light pen response as correct, incorrect 
or undefined. Responses m^ also be entered through the keyboard, however, no ^ 
has been made of this response mode in the reading program. Obis is not to 
the value of keyboard responses but rather to admit that we have not 
as yet addressed ourselves to the problem of teaching first-grade children to 
handle a typewriter keyboard. 

Prerecorded audio messages are played to the children through the earphones. 
There is also an on-line recording capability. The children may, when the 
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There is also an on-line 




microphone Is activated, record their osn production of a given text displayed 
on the screen. This can then he plsyed back to them with or without an adult 
model. The recording and pl^-hack c^blUty helps cos®ensate for the absence 
of a voice analyzer. The system cannot evaluate the student's vocal production 
and therefore each student becomes his own voice analyzer. 

The l6 student response terminals are serviced hy an Iffl l800 Process 
Control co»®uter. This central processing unit has a relatively limited (l.e., 
32K) imedlate access core storage. Rapid access bulk storage Is provided by 
six Interchangeable disk drives, each disk containing 512,000 sixteen bit words. 
The audio coitponent of the system consists of a bank of IM 1505 audio units. 
Each audio drive unit Is connected to one of the response terminals but the 
connections can be 'raried at will. Response data flowing Into the system from 
the student terminals is recorded on two I» 2U02 tape units, to IM 1501 
station control, a lUU2 card reader punch and a HA3 Uae printer co«®lete the 
configuration of the IB4 1500 CM system. 

Insert Figure 1 about here 



Time -Sharing . 

The IM 1500 CAI system is a time-sharing system in the sense that the 
activities of each student response terminal are examined in sequence and appro- 
priate actions taken. The time-sharing Is not strictly sequential in that an 
Interrupt feature allows certain priority conditions within the instructional 
system to alter the sequence of programmed instructions. 
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A non-tectailcal characterization of the flov of Infoniatlon In the system 
-ay serve to suggest the operation of the time-sharing system. Assume that ue 
are breaking into an Instructional session and that the system Is preparing to 
examine a response entered hy a student at terminal 1. The appropriate terminal 
record and lesson Instructions are read from the disk storage and placed In core. 

. The response coordinates from terminal 1 are evaluated and c«npared to those 
stored in the lesson progrma for the given problem. (See the section on lesson 
coding for a detailed discussion.) Decisions are made on the basis of the lesson 
program logic and appropriate commands are given: 1) to the station control to 

display certain text on the CKT, 2) to the film projector to position and display 
a certain frame, and 3 ) to the audio unit to play track a segment n to segment 
n+p. The attention of the system then moves to terminal 2 ana the process Is 

repeated. 

The above description Is almost aUegorlcal In Its overslu^Uflcatlon. It 
is intended only to give a general feeling for »hat is meant by timesharing on 
the system. For a complete technical discussion the reader Is referred to CAI 
Programming Systems Users Qulde, Preliminary Draft, IM, Armonk, Hew Yort, 1967- 
The entire process moves at a very rapid rate. Subjectively, the student 
at the terminal feels that he has the fuU attention of the system. The response 
time for the CRT Is less than one second and the projector response is nearly 
as fast. The audio response time is someuhat slouer, ranging from 2-U seconds 

on the average. 
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SCHDOL-LABORi\a»RY ORGiJ«IZAPION 

Sc^f>ol Populatilon * 

The Brentwood School, a K-6 elfimentary school, is located in the Bawenswood 
City School Mstrict in East Palo Alto, an nninconorated section of San Mateo 
County adjacent to the City of Palo Alto. The population of the Brentwood 
School is apprexinately 80 percent Begro with the remaining 20 percent about 
evenly divided between Mexican-American, Oriental-American and Caucasian. The 
school quaUfies for federal aid to Iji^overtshed areas under Title I. 

The Stanford CAI Project is concerned with the first-grade students at 
Brentwood School. There are four first-grade classrooms at Brentwood, two of 
which receive instruction in mathematics under conqaiter control and two receive 
instruction in reading. Of the two classrooms in the reading program, the student 
population is approximately 90 percent Megro, 6 percent Mexlcan-Amerlcan, 2 per- 
cent Oriental-Anerican, and 2 percent Caucasian. 

PriniftTy-grade Organization . 

The students are grouped in first grade at Brentwood according to billow 
maturity. The grouping is made on 're basis of recommendations hy the kinder- 
garten teachers and a reading-readiness test designed by district personnel. The 
reading project works with both a high maturity and a low maturity room. 

Both of the teachers involved in the stanford-Brentwood CAI Beading Project 
are experienced first-grade teachers; one has been teaching in the Ravenswood 
District for l6 years, the other for 7 years. 

Preparation for Innovation . 

A serious attea^t was made on the part of the Stanford Project to prepare 
the teachers and parents of the Ravenswood District, and of the Brentwood School 
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in particular, for the acceptance of the technological innovation of a cooputer- 
asslsted instructional laboratory. A good deal of effort vas expended during 
the school year 1965-66 in this preparation. Workshops were held for the Brent- 
vood personnel and an extension course vas conducted ly the project staff for 
the district personnel. In both the 'vorkshops and the extension course discussions 
. centered on the general concept of programmed learning, linguistics and reading, 
time-sharing on the computer, a description of the IBl 1500 system, and a very 
thorough discussion of the re.ading curriculum. Besides the workshops and the 
inservlce training courses, weekly meetings were held with the principal and 
primary grade teachers at Brentwood to discuss in greater depth the plans for 
the ensuing year. During the summer of 1966 a two-week workshop was held on 
the Stanford campus for the two teachers who would be directly involved in the 

reading program. 

Ho attempt has been made on the part of the Stanford project to dictate 
in any way the on-going reading Instruction within the classroom. Rather, we 
l«ve viewed the complete initial reading program as a shared responslblUty based 
c informed cooperation between the project staff and the classroom teachers, 
ekly meetings are held between the teachers and the project persoi^l to evalu- 
i,e the students' progress on the system and to exchange views and information 
- ^out both the classroom and the laboratory instruction, and the performance of 
'le students ’.n both environments. 

A of the Stanford staff is permanently placed in the Brentwood School, 

. otlng as a school-affairs Ualson officer. His duties include the resolution 
ct any organizational problems that arise in the running of the CAT project within 
1 le Brentwood School and also the handling of large numbers of visitors who have 
.rited the laboratory during the year. 






The preparations and the current efforts for school-project cooperation 
hare paid off handsomely. The enthusiasm and support of the teachers has been 
highly gratliving. We hare also held many exhibits, open houses, and discussion 
groups for the parents of the Brentwood School and the patrons of the Harenswood 
School District. Again, the support of the parents, the school board, the admin- 
. istration and the teachers has exceeded all expectations. 

Laboratory Orp^anlzatlon . 

The laboratory Is housed in a rectangular prefabricated steel structure 
approximately 3200 square feet in area, located on the Brentwood School grounds. 
The building contains the tereinal room, the off-line teaching room, the central 
con?uter room, and a group of offices for rhe laboratory personnel. 

The staff of the Stanford-Brentwood CAI laboratory consists of ten members. 
The laboratory is under the general management of a senior programmer who is also 
in charge of the data reduction staff. His staff Includes two programmers, two 
graduate students and a secretary who also functions as a receptionist. The 
systems group is headed by another senior programmer who has on his staff an 
assistant programmer and a cosg-uter operator, plus a technician v*o handles audio 
assemblies. The coding groups for both the mathematics and the 'reading program 
are also housed in the laboratory. The reading coding group is directed by a 
senior lesson programmer and consists of four coders, plus part-time debugger 
and graduate assistants. A similar number of personnel comprise the mathematics 
coding group. Three proctors handle the children in the terminal room itself 
and are responsible for off-line instroctlon. Iffi has also provided several 
customer engineers who are either on duty at the laboratory or are on call, 
present the stability of tl« system is such that the staff of IH4 customer 
engineers has been reduced to a single man during school hours. 



Laboratoi^ Operation. 
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Full scale operation with the students began on the first of Ifovemter, 1966. 

The starting date was somewhat later than originally anticipated, due to a delay 
in the delivery of the 1500 system which arrived at the Brentwood School on July 10. 
The ensuing Interval from July 10 to Hovemher 1 was devoted to a shahe-down and 
.debugging of the system which was, of course, untested In on-Une operation. 

on the first of Kovemh - the students began coming to the laboratory on a 
daily basis. The students received a week of orientation lessons outside the 
terminal room in the off-line teaching room. During this period they were 
acquainted with the use of the earphones, microphone, and the light pen. Exer- 
cises and games were carried out which acquainted them with the equlpsent and 
the kinds of learning tasks that they v»uld encounter in the terminal room. Iteing 
the second week in November they were introduced gradually to the actual response 
terminals on a staggered basis; that is, a lesson in the terminal room was fol- 
lowed by a lesson in the off-line teaching room. ^ November 15 the students 
were on the system dally for a 20-minute instructional period. During these 
early exposures to the terminal equipnent an adult was stationed behind each 
Ohild to assist him with whatever problems he might find in handling the equiprent. 
It was soon evident that the children were adapting quickly to their new environ- 
,«nt, and the adults were gradually withdrawn. By mid-November the proctor staff 
„ithin the terminal room consisted of the same personnel as it does today 
caching proctor, one machine proctor and a ren«dlal reading teacher. 

The students come to the laboratory in four groups, since the terminal room 
equipped to accommodate only one half of a normal classroom at a time. Each 
, ,^up receives 20 minutes of instruction and there is a minimum of 10 minutes 
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between groups. The 10-mlnute interval is necessary in order to prepare the 
system for the next group. The headsets and light pens are swabbed with alcohol 
to minimize the risk of infection. Each student is signed-on at a response 
terminal the appropriate films and audio tapes are loaded into the projectors 
and audio-drive units. When the sign-on process is complete the name of the 
student assigned to a given terminal appears on the CRT. 

When a new group arrives, the students enter the terminal room, seat them- 
selves before their assigned terminals, put on their earphones, touch their names 
with the light pen and the lesson begins. During the instructional session the 
machine proctor is stationed at a proctor’s typewriter which transmits messages 
from the central system. Tliese messages consist primarily of identification of 
some system or terminal failure, transfer of a student from one lesson to the 
next, or notification of an error limit exceeded at some terminal. In the case 
of system or terminal failure the machine proctor takes appropriate action, 
which may range from transferring a student to an empty terminal to notifying 
the head systems programmer. When a student transfers to a new lesson the machine 
proctor must change his audio tape. Each block of problems within a lesson has 
an error limit which is 50 percent of the total number of problems •vrtthin the 
block. If this error limit is exceeded by the student, notification of this con- 
dition is transmitted to the machine proctor, who in turn conveys the information 
to the teaching proctor. The teaching proctor then observes the student and 
chooses between two courses of action. If the difficulty appears to be mechanical 
(e.g. , the student is not using the light pen correctly) or some minor misunder- 
standing of directions or lapse of attention, the teaching proctor may sign on 
the terminal with the student and help him through the troublesome section. By 
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signing on the terminal, a proctor bit is set in the response record to identify 
responses which are not necessarily those of the student. If, however, the 
student's difficulty appears to be of a serious nature, the proctor may remove 
the student from the terminal and transfer him to the remedial teacher for 
diagnosis and personal instruction in the off-line teaching room. Fortunately, 
•during the entire year's run the necessity for such off-line instruction has 
been very infrequent. The remedial reading teacher has functioned primarily as 
an assistant to the teaching proctor within the terminal room and has been brought 
into play in her major function only when the system has gone down or an individual 

teminal has failed. 

Both the teaching proctor and the remedial reading teacher hold elementary 
teaching certificates from the State of California and are experienced classroom 
teachers. The machine proctor is a member of the lesson programming group. The 
children are accompanied between the classroom and terminal room ly an assistant 
proctor who performs certain clerical tasks while the students are on-line. 

Since the two first grades in the reading program are part of the ungraded 
primary plan at Brentwood, a certain amount of transfer of students between the 
low and high maturity rooms has taken place in the course of the year. The 
of the students from one group to another has beer, one of the ma^or 
topics of the weekly meetings between the first-grade teachers and the Brentwood 
project staff. We have been able to accommodate all requests by the teachers 
for exchanging students within groups. Our only stipulation has been that the 
exchanges be made known to the project staff by Thursday of the week preceding 
the actual change of students from one group to another. The lead time has been 
necessary on our part in order to keep the student registry pack in proper order. 
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The amount of migration of students to and from the Brentwood School has 
been relatively small, at least for the two classrooms Involved in the CAI readli 
project. Pour of the original students have left the school and have been re- 

placed l3y four incoming students. 

The population of the first-grade classrooms was somewhat smaller than 
. anticipated for this year, which left terminals unoccupied for each of the four- 
groups. During the early phases of the program this arrangement was fortuitous 
in that spare terminals were available for transferring a student when his origl 
failed. After Easter vacation it was felt that the system had achiever 
a degree of stabiUty, however, which no longer required a large number of termJ 
to be available as bactarps. The e«®ty terminals were then filled with an appro- 
priate number of remedial second-grade students. Since they started late in th. 
year their performance will not be considered in this report except to note in 
passing that the second-grade teachers Involved report a definite increase In 
interest and application to the reading task within the classroom. 

Laboratory-Classioom Cooperation . 

It would be unrealistic in the extreme to claim that all Initial reading 
instruction is being carried out in a 2C-minute session on the CAI system. As 
stated previously, we view the program as one of cooperative effort between th< 
normal classroom reading instruction and the Stanford CAI project. Accordlngl: 
we have acquainted the teachers with the CAI curriculum and have Issued regula: 
reports on the performance of their students on the system. We have asked, in 
return, only that the classroom teachers keep us Informed of the kind of Instr 
tlon they are carrying out within their own rooms. This dialogue is effected 

during our weekly meetings. 
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Both classroom teachers started the year with an IndlrtduaUted reading 
program. However, after the Christmas vacation a more structured basal series 
approach was Introduced. From approximately md-February to the end of the 
an eclectic approach was used. Incorporating features of both the basal series 

and tbe indi'/idualized program. 

The project's contribution to the dialogue consists primarily of a weekly 
report which Is generated by the computer and which gives the student location 
hy lesson within the curriculum, plus a wel^ted Index of each student's per- 
formance within each of tbe standard lesson blocks. This performance Index Is 
cumulative but is weighted for current performance mors .heavily than for past 
performance. Also Included on the weekly report Is a record of time lost from 
the system due to various causes and a cumulative total of each student's time 
on the system. On the basis of the weekly report and discussion within the 
eeekly meetings, the classr«m. teachers are able to devise and carry out Instruc- 
tional procedures appropriate to each student's progress on the system. 

Ideally, a CAI system such as this would have available for the classroom 
teacher a large set of closely correlated materials. Efforts have been Initiated 
toward this goal, to extensive teacher's manual Is being written wMch will in- 
clude not only a detailed description of the curriculum but also a large number 
of correlated and weU prepared classroom activities from which the teacher may 

e VI ovmeae h® Tcfivcd to tlocks of lessoiis aud vill be 

choose. Exercises and games vill oe xeyea w uaw 

applicable to smaU groups as weU as to individual students. Tbe manual, at 
least in Its elementary form, wlU be available for use during the school year 
1967-68. Both of the classroom teachers Involved In the reading project for thl 
past year will assist in the design and writing of the manual during the su»«r 

Of 1967. 
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CURRICU£JJM RilfflONALE 

A basic sss^ptlon underlying the Stanford Cffl Beading Curriculum is that 
the EngUsh speaking child brings to the initial reading task a relati«ly large 
vocabulary and at least an operational knowledge of EngUsh syntax. He has a 
knowledge of the language which is sufficient to enable him to coimmmicate with 
hi- peers and with adults. Therefore, the primary goal of initial reading is 

to teach the language but to teach the orthographic code by which we represent 

our spoken language. 

If one is attempting to teach a code, the most reasonable approach is to 

begin, not with aU the irregularities and exceptions, but rather with the gu 

and consistent patterns. This jxrsltlon is not unique to the Stanford Project. 

It has been advocated by Unguists for some years (Bloomfield (1942), Fries ( 1963 )) 

and has been is^lemented in several -Unguistically oriented" reading series. The 

sequencing of monosyllabic patterns (Table l) and certain extensions and refine- 

r^rrts of the basic notions as stat^ by tte aixrve-mentioned authors, constitute 

the original work in curriculum design carried out by the Stanford Proj 

A detailed discussion of the psycholinguistic rationale of the curriculum 

may be found in Bodgers ( 1967 ) =“>3 Hansen and Bodgeis (1965)- Tte following 

stations from Bodgers (1967) provide a concise su-.ar>- of our goals and procedure 

From a practical point of view, our p^ram is m 
^rortde non-readers with some limited f®lytic ski^ 

phonological, morphological, ^ S skills, 

some considerable confidence in tte use of ttese sl^ls 

It is not our intention to teach the 

sound-symtol P^**-”* ^^"i^eq^nt vSabulary 

itLs, or all of the sentence patterns 

rMiT. iitention to give the student enough skill and sen 
o^flSn^ to involve him in that confrontation kmwn as 
Sg™ fading! we believe it is the ability to make 
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reasonable 

sequences on the printed pagp tnax w *u^ 
to teach in reading. 

We haw deftned the Stanford ^roach to 

as applled-psychoUngulstl^. ^ 

basis of linguistic infoimation about the struc 

5^, e:Sncal o^«'^«°“^,"^,:^."“TLrh^theses 

Z^t^L^teT^Tn 

to represent as rcuxxstically ^ SsL of experljnental findings, 
and teaching situation^ On retested and \atimately 

dictating 

incorporated into tne cui-xx^, ^ course, sane- 

presentation variables an va i^-ttle curriculum material 

cn the basis of Edified or re-foiraulated 

probable that ^se e«o^t of empirical 

on the basis of the considerab^ ® I ! of a vear's CAI 

evidence vhich will be available as tne re^lt of a year . u 

experience with classes of beginning readers. 



CURRICULUM IfcSCRIPTIOlI 

The Stanford CH Reading Curriculum may be divided into four broad areas 
,f concentration: 1) decoding shills, 2) compiebension, 3) games and other 
.^tivational devices, and U) review. The lesson materiel and teaching strategies 
:wiU be discussed briefly in each area. While all lessons are not alUe in the! 
cquence of events, the block level flow chart (Figure 2) of Imvel 111, tesson 9, 
-av be considered as a fairly representative exea^lfi. 




Insert Figure 2 about here 
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1. Decoding Skills 

Letter and Letter-String Identification . 

No direct attempt Is made to teach the names of the letters. From the 
point of view of our program, letter names are In many Instances at odds with 
the dominant sound which the letter represents. Letter names are used In the 
audio messages hut the emphasis Is on discrimination and recognition of Individual 
letters and letter- strings In varying orthographic contexts rather than on 
mastery of the letter names themselves. 

Single Lf^tter Matching . A model letter Is presented on the projector and 
two or three letters are presented on the CRT. The wrong alternatives are deslgne 
to Include letters considered to he easily confused due to, their similar ortho- 
graphic features. The student Is requested to choose the letter on the scope 
that matches the model presented on the projector. 

Letter-String Matching . A model string o:' two or more letters Is presented 
on the projector and a choice of two or three strings Is presented on the CRT. 

The student Is requested to choose the alternative that matches the model string 
on the projector. The alternative letter- strings Include reversals, one letter 
differences, two letter differences, easily confused letters, and differences 
In serial order. 

Same-Different Task . A pair of letter- strings Is presented on the scope. 

The student Is requested to touch one of two boxes that designates whether the 
letter-strings are the same or different. The letter- strings are designed to 
Include reversals, one letter differences, two letter differences, and easily 
confused letters. This task provides additional opportunities to acquire func- 
tional recognition ^ues and the serial order concept of letter sequences within 



words. 
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Word List Learning . 

This section of the curriciaum may be described as a set of paired-associate 
tasks where the stimulus is the verbal pronunciation or pictorial representation 
of a word (or both), and the response is the correct identification of the appro- 
priate written word in a list of written words. The lists for any given lesson 
are composed of words generated by the rhyming and alliterative patterns being 
presented in that lesson* 

Quantitative learning models, appropriate to the paradigms in this section 
but similar in nature to those existing for classic paired- associate learning 
(Atkinson, Bower, and Crothers, I 965 ), may be developed to describe acquisition 
processes to assess the effects of learning and forgetting. 

Five problem types (PT) are included in this section with each problem type 
containing approximately six problems. Each PT represents a step in a cue-fading 

technique. The five PT's are: 

PT 1 - Cues: Picture, Orthography, and Audio 

PT 2 - Cues: Picture and Audio 

PT 3 - Cues: Audio only 

PT U - Cues: Picture only 

PT 5 - Criterion Test. Cues: Audio only 



The student responds to a set of cues y touching a word in a list of words 
on the CRT. Each student response receiver immediate feedback. If a response 
is correct it is reinforced. If it is an error the correct answer is indicated 
by an arrow and an overt correct response is required before the next problem is 
■! e •K-r.Qnr.Vi*»ri t.n a-maroDriate remedial problems. This is 



er|c 



presented, or the student is 





Flow Diagram for Word List Block 
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a general teaching strategy which is adhered to throughout all the lesson materials 
except in the case of the criterion tests. PT 5 is a criterion test and is similar 
to PT 3 except that no feedback or correction is given on the first presentation 
of the list. If the student meets criterion (l.e., no errors) on the first cycle, 
he leaves the word list presentation block. If he does not meet criterion, he 
returns to the task described in PT 2, branches past PT 3, and is presented with 
those items in PT k which he missed in PT 5- He then makes a second cycle through 
PT 5 with correction and optimization to one initial correct response for each 

item. 

The problem route for these five problem types is represente(= by the flow 
diagram in Figure 3. Correct response routes are indicated by solid lines and 
remedial branches by dotted lines. 

Insert Figure 3 about here 

The optimization routine referred to in Figure 3 cycles the student through 
the list of response requests, repeating only those items which were responded 
to incorrectly until each item has received one initial correct response. 

pppiPdial Materials for Word List Presentation Sections . 

Early studies designed to test the feasibility of the lesson material indicated 
that the performance of a small percentage of the students was lower than antici- 
pated on the word list materials. It was difficult at that time to specify all 
of the problems of these slow learning students since the learning difficulties 
were varied and complex. However, the learning problems can be grouped into four 
categories : 



O 
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1) A need for finer discrimination of initial or the final units of 
stimulus words either as letters or as sound patterns or both. 

2) A need for practice with fewer words. The memoiy processing 
capacity of some students seemed to be over- taxed. 

3) A need for extensive practice on given words that troubled a 

particular child. 

4) A need for a richer semantic context within which to relate the 
new words being learned. 

Four sets of remedial materials to supplement the word presentation section 
were developed to give additional specific practice to those students experiencing 
one or more of the above problems. The four remedial sets are called: l) Letter- 

Word Saliency, 2) Alliteration and Rhyming, 3) Rich Sentences, and U) Redupllca- 
tion and Sentence InJ.tiator. 

The four blocks of remedial word presentation materials are sequenced after 
FT 1 in the word list presentation block. It will be recalled that in a con- 
ventional sequence for the word list presentation block in FT 2 (Figure 3) , 

Insert Figure 4 about here 

the picture is presented on the projector without the printed word, and a touching 
response to the appropriate word on the scope is requested. A counter is set at 
the beginning of FT 2 so that it tallies the errors for each initial presentation 
cycle but not for the optimization routines. At the end of the first cycle, the 
counter is examined. If its contents exceed a predetermined value (e.g., 50 
percent of the number of response requests) the student is branched to the first 

•.issigned block of remedial material. 













FAIL 1st CYCLE (CA> 50%) 




Figure 4o 

Flow Diagram for Word List Block with Remedial Units 
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After completion of the remedial block, the student is returned for another 
cycle through PT 2. The error counter is again set to zero and the above operation 
is repeated. If the student again has an error rate of greater than 50 percent, 
not including the optimization routine, he is branched to the second assigned 
block of remedial material. The branching process is repeated if necessary until 
all four blocks of remedial material have been exhausted. In the case where a 
student completes a fifth failing cycle in PT 2, he is branched out of the in- 
structional material and the proctor is automatically called. It is possible 
then for a student to receive from zero to four remedial presentations according 

to the evaluation after a cycle through PT 2. 

If a student meets the criterion for Problem Type 2, he proceeds through 
Problem Types 3, k and 5, to the word criterion test. The decision counter is 
again set to zero at the beginning of the word criterion test. If a student 
meets criterion he proceeds to the next section of lesson material. If his 
performance in the test does not meet criterion and his error rate is less than 
50 percent of the number of response requests, he is branched to PT 2. If, 
however, the error rate exceeds 50 percent, he is branched directly to the next 
of the remaining remedial sections before being recycled through Ff.2, k and 5- 
In a second cycle through PT 5, the student is given an optimization routine 
and allowed to proceed to the next section of the curriculum. Therefore, there 
are two ways by which a student can reach the remedial material. The first is 
to exceed the error criterion for PT 2 and the second is to exceed the error 

criterion during the first cycle of PT 5* 

The same general procedures apply to eicher method of entering the remedial 

material. It must be noted, however, that only one entrance to the remedial 
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.aterisl ms. .e Made fM,M vora list criterion test since the second pass 
through tx» criterion test is ^rely an optimization routine with no error counter 
being set for second branching to remedial loops. An evaluation of the remedial 
Material and the remedial sequence vill he made in the folloving manner. For 
each »ord list presentation section (that is to say, for each lesson), the four 
blochs of remedial material have been randomly ordered. The response data of 
students receiving remedial material will he subjected to off-line analysis ir. 
an attempt to discover if the increments in performance following each of the 
remedial blocks differ significantly from each other. 

Matrix Construction . 

certain rhyming and alliterative word patterns can > considered re^lar; 
that is, certain graphic sequences correspond without exception to certain sound 
sequences. Fries (1963) and Bloomfield (19^2), among others,.have suggested 
that such patterned word regularity may be a key concept in learning to decode 
English print. This decoding concept is implemented in the Stanford materials 
via the sequencing of monosyllabic patterns as shown in the Vocahu.l. -quen-.e 
Chart (Table l). The sounding matrix is an instructional technique .V, ■ liows 

for practice in learning to associate orthographically similar sequence, ...l.n 
appropriate rhyme and alliteration patterns. 



Insert Table 1 about here 
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Rhyming patterns are presented in the columns of the sounding matrix. 

an 



r 



f 



c 



Alliteration patterns are presented in the rows of the matrix. 





an 


at 


as 








r 


ran 


1 rat 


rag 



The matrix is constructed one cell at a time. The initial consonant of a 
consonant- vowel-consonant (CVC) word is termed the initial unit and the vowel 
flnti the final consonant are tenned the final unit. This division is maintained 
later when the words are composed of consonant clusters and diphthongs. The 
intersection of an initial unit row and a final unit column determines the entiy 
in any cell. 

The problem format for the construction of each cell is divided into four 
parts: Parts A and D are standard instructional sections and Parts B and C are 

remedial sections. The flow diagram (Figure 5) indicates that remedial Parts B 
and C are branches from Part A and are presented independently or in combination. 

Insert Figure 5 about here 

The following example of a matrix cell construction is presented in sore 
detail in order to highlight the instructional sequence and remedial branches. 



ran 



fan 



can 
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The studcrt eees the empty cell with Its associated initial and final units and 
an arr^' of response choices. He hears the audio message indicated by response 

request number 1 (BR l) Figure 6. 

Insert Figure 6 about here 

If the student makes the correct response (CA) (l.e., touches ran) he procee. 
to part D where he sees the word written in the cell and receives one additional 

practice trial (Figure ?). 

Insert Figure 7 about here 

In the course of an errorless trial the student hears the word pronounced 

three times and he must identify and pronounce it twice. 

In the initial presentation in Part A, the multiple choice items are de- 

signed to identify three possible classes of errors: 

1) The init. al unit is correctly identified hut the final unit is not. 

2) The final unit is correctly Identified but the initial unit is not. 

3) Heither the initial unit nor the final unit is correctly identified. 

If, in Part A, the student responds with ^ he is branched to_ remedial 

section B where attention is focused on the initial unit of the cell (Figure 8). 

Insert Figure 8 about here 

If a correct response is made, the student is returned to Part A of the 
problem for a second attempt which will be optimized for a correct response. If 
an error response (WA) is made in Part B, the indicator is displayed above the 
initial unit beside the cell at the point indicated by the first circled super- 
script @ in the audio message and held until the second superscript (£) 








Figure 5 ® 



Matrix Flow Diagram 



an 



rat 

bat 

fan 

ran 



an 



ran 



RR 1: Touch and say the word that 

belongs in the empty cell. 

CA: (Branch to Part D) 



WA 1: 



WA 2: 



No, 




rat = final -*C, -* A 
fan = initial B, A 
bat = other B, -»C, 



ACA 

No, touch and say ran. 



Figure 6. 

Matrix Problem, Part A 



RR 1: Good, you have put rcm in the 

cell. Touch and say ran. 

CA: Good, ran. (-»next problem) 

iCA 

WA: No, touch and say ran. 



Figure J. 

Matrix Problem, Part D 
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At the completion of the response request, the indicator is placed beside the 
correct response area and held until a correct response is made (^A). 

in Part A, the student responds vith rat , he is branched to remedial 
section C where additional instruction is given on the final unit of the cexl 

(Figure 9)* 

Insert Figure 9 about here 

The procedures in Part C are similar to those of Part B. It should be noted 
in the remedial section that the initial letter Is Lever pronounced (Part B) 

whereas the final unit is always pronounced (Part C) . 

If, in Part A, the student responds with he is branched through both 

initial and final unit remedial vork. 

When the student returns to Part A after completing a remedial section, a 
correct response will advance him to Part D as indicated. If a wrong answer 
response is made on the second pass, the indicator arrow is placed beside the 
correct response area and held until a correct response is made. If the next 
response is still an error, a message is sent to the proctor and the sequence 

is repeated from the beginning. 

When a student has made a correct response on Parts A and D, he is advanced 
to the next word cell of a matrix which has a problem format and sequence identical 

to that just described. 

The individual cell building is continued until the matrix is complete. The 
majority of the matrices in the lesson material contain from six to twelve words 
and nonsense syllables. Nonsense words are considered legitimate cell entries 



xmder the following constraints: 



1) Nonsense items are oc current English syllables. 

2) Honsense items are not used vhlch represent "regular" but unconventional 
spellings for common monosyllabic words. For example, although sed 
represents a regular speUlng for the initial English syUable in words 
such as sediment , it would iot be presented in matrix format since it 
would be considered an unacceptable spelling for the homophonous mono- 

syllabic word said . 

3) The proportion of nonsense words in the matrix is less than UO percent 
of the total cell entries. 

When the maurix is complete, the entries are reordered and a criterion test 
over all cell entries is given (Figure 10) . 

Insert Figure 10 about here 

Randomized requests are made to the student to identify the cell entries. 

Since the first pass througli the criterion matrix is a test trial, no reinforce- 
ment or correction is given. Errors are categorized as initial, final, and other. 
Remedial exercises are provided for both initial and final errors. The branching 
procedure is similar to that in the cell construction section with the exception 
that, whereas the branching in the cell construction block is contingent on each 
separate response, the branching in the criterion test is contingent on the total 
performance in the test. If the percentage of total errors exceeds some specified 
criterion percentage (e.g., 20 percent) of the total responses, the category 
registers are examined. If all the errors are recorded in one category (Initial 
or final), only the remedial material appropriate to that category is presented. 

If the errors are distributed over both categories toth types of remedial material 
are presented. This material is highly slmll.-r to conventional phonics exercises. 
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The standard optimization scheme In which an Initial correct response Is 
obtained for each Item Is used for all sections of remedial material. 

After uorklng through one or both of the remedial sections, the student Is 
looped back for a second pass through the criterion matrix. The second pass is 
a teaching trial as opposed to the Initial test cycle; the student proceeds with 
the standard correction and optimization routines. 



Compound Words . 

The approach to compound words assumes the existence of a learning-transfer 
process in which the child knows how to read one of the two elements that form 
the compound word. The learning task consists of reviewing this known word and 
learning the unknown part of the compound. In addition the child must comprehend 
the conventional meaning of the compound word as well as Its role In a fairly 
rich sentence. Thus we are utilizing confound words to study the degree to which 
children can transfer partial reading mastery (l.e., recognition of one of the 
two words In the compoimd) to a variety of reading contexts. 

Compound words are Introduced In Level III and are Initially composed of 
two known monosyllables (e.g., ^ and m^ are mastered prior to the presentation 
of the compound batman ). Sequences are then Introduced comprised or five com- 
pound words of which only one of the elements Is known. Words are selected 
according to the following three criteria; 

1) frequency in initial reading materials, 

2) imaginative possibilities so that semantically rich context sentences 
may be written, 

3) the opportunity to vary the known word in initial and final position 
in the five compound words (e.g., hat b ox, firehat, h^band, etc.). 
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PoU^syllablc Words . 

Our approach to non-oompound polysyllabic words focuses primarily on the 
role of stressed syllables in their relationship to verbal rehearsal and memory 
processing. We hypothesize that polysyllabic words with stress on the first 
syuable and a reduced or neutral vowel (usually transcribed as schwa (a)) in 
the final syllable will be most easily learned. In the Stanford Heading Cur- 
riculum the acquisition of polysyllabic words is, as with compound words, a 
transfer learning process in which the first syllable of a polysyllabic word 
follows the pattern previously taught in the word list and decoding exercises. 
The stress and the number of syllables Involved in the sequence are tightly con- 
trolled. The precise roles for sequencing polysyllables are as follows: 

11 the words are, at first, blsyllablc, 

2) the first syllable of the word is stressed, 

3) the vowel of the final syllable is represented as phonemic /a/, 
excepting /±Y) / , /liy/ > /iy/ * 

It) the first syllable is regular in that it follows the pattern learned 
in the word list and matrix exercises, 

5) the word is monomorpihemlo where possible, and 

6) the words are found in the Lorge-Thomdlke Word List (19^). 

Utilizing these rules, the unstressed second syllables shown in Table 2 

were added to an appropriate vocabulary item to form a polysyllabic word. 



Insert Table 2 about here 



Polysyllabic words are Introduced in level IV. First presented are double 
-hvmes in which ■is’ - -e -• ■ iv-n lesson are both graphemically and 




Table 2. 

Unstressed Second Syllables 
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ar en ip ot 
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er it ow 








et 
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phonemically aiiVe except for the initial consonant (e.g. , bitter, litter, etc.). 
Subsequently, vords are presented in which the second syllables rhyme and repre- 
sent moiphemically similar elements (sapling, starling). In general, polysyllabic 
words that are of high frequency and are useful for the story material were 
selected. We assume the words are understandable to the children, although no 
■ semantic criteria were specified. 

The sequence of polysyllabic vocabulary al bows for a build-up from two to 
three syllable words. There are direct syst«anatic variations given between 
monomoiphemic and polymorphemic word entries. All cf these variations will be 
evaluated for their influence on the learning process. 

The lesson structure for polysyllabic words consists of a sequence of approx- 
imately five words. For each pattern type (double rhyme, second syllable rhyme, 
morphemic or non-morphemic second syllable) the child is given a screening test 
similar to the matrix screening test. The child is asked to touch the cell 
where the two elements joined together such as 1^ + der forming ladder belong. 
Should the child be unsuccessful, he is presented with a rhyme utilizing the 
words from the screening test, followed by sinple phonemic -grapnemlc decoding 
exercises adjusted to that particular pattern of polysyllables. The child 
performing successfully on the screening test, and the child who has been presented 
with the phonemic-graphemic exercises then proceed to a lesson utilizing two 
words from the screening test and three to four others which follow the pattern 
iieing presented. 

Sentence Initiators . 

In reading writxen discourse, our learning objective is to train the initial 
readers to pronounce sentences with conventional intonation. A conmon reading 






o 
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tehavior for beginners is giving "final word stress" to each of the words in a 
sentence. We wish to teach the child to adjust his timing, pitch, and stress 
contours so as to read sentences with intonation patterns commonly found in 
speech. This learning task takes on significance when one becomes aware that 
our CAI system does not include a speech analysis device nor is such a device 



even technologically available at this time. We depend upon our initial reader 
to be his own speech analyzer in this section of the Stanford Re*jding lessons. 

We do have available one of the in^ortant features of any CAT system, namely, 
the precise control of the timing of all learning events. The virtue and logistic 
nuisance of a CAI system is the required specification of all the time intervals 
for each of the instructional and reinforcement events within every problem. In 
this, as well as other sections of the lessons, we utilize the pacing feature of 
the CAI system to accelerate the reading- speech output of the children and, con- 
sequently, cause the children to approximate the speed and rhythms of normal 

speech while reading. 

We maximized the expectancy of conventional sentence intonation by selecting 
hi^ frequency sentence conn?onents such as "It's a," "That's a," "They can, etc.. 



that are utilired comnonOy In the introduction of high frequency nouns and verbs 
in speech. These high frccPiency initiating sentential strings were selected 
from Carterette's (1965) list of multi-uord units uttered hy six-year old children 
during a free discussion session. It is to be noted that these vord strings are 
pronounced by the children as if they vere polysyllabic wjrds. To each sentence 



initiator ve attach a noun or a verb the children have previously learned in 
the list and matrix sections of the lessons. This provides us with an opportunxty 



to review these words. 
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After a number of pilot investigations focused on the timing of the lecuning 
events j ve found the following experimental procedure to be most promising in 
teims of achieving our objective. The sequence of the procedure Is as follows: 

1) anticipation Interval ; a sentence appears on a display device and 
the child Is given two seconds with which to attempt an oral reading. 

2) reinforcement Interval ; the audio device then plays a reading of the 
sentence to the child. 

3) modeling Interval ; the child Is given two seconds to repeat the reading 
of the sentence. For eiwh new sentence the cycle Is repeated. 

In reinforcement tezms, the child Is es^iected to use self-evaluation In order to 
note any discrepancies between his reading and the machine's reading. 

Each exercise in the "sentence Initiator" section contains 23 sentences. 

Each Initiator Is presented eqpproKlmately five to ten times In a given lesson 
and for a total of 100 to 125 esqposures In all the lesson^. At this point , the 
sentence Initiator Is utilized In the story material. 

The manner of presentation of the sentence Initiators allows the student 
to monitor his own production hy comparing It with the audio model. Mdltlonal 
monitoring of the student's perfomance In the sentence Initiator material Is 
accomplished by the proctors and by sample recordings made on a sex>arate recording 
system. 

2. Comprehension 

Producing the appropriate verbalization when confronted with some orthography 
(decoding or word-attack skills) Is a necessary component but not a complete 
definition of reading. An equal and ultimately more Important aspect of reading 
may best be t< - ...ed "comprehension.” Although a great deal has been written about 
"reading comprehension," It Is not at all clear what we mean tiy the tezn. The 
general approach In current reading materials and reading achievement tests is 
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at the paragraph level and primarily employs practice in recall or identification 
of specific details and sequence of events and in Identifying the "main idea. 

While using the standard techniques, «e have also attempted to look at the 
question at the sentence level. Although ue make no claim of achieving a complete 
definition of the field, ue advance three propositions as necessary components 
of such a, definition. In order to maintain that one comprehends a sentence one 
■nst be able to demonstrate that one has: 1) an appropriate set of semantic 
associations for each lexical item in the sentence, 2) an operatio n knowledge 
of syntax, and 3) the ability to identify the lexical items in the sentence 

which convey a given piece of information. 

The next three sections of lesson -raterial to he described are designed 
primarily to gather data pertinent to the above propositions. They are not, in 
the strict sense, teaching sections except that, as is the case throughout the 
lesson material, an overt correct response is required for each problem before 
the next one is presented. We do not feel that enough is known about compre- 
hension to justify elaborate branching, remediation and optimization routines. 
Hopefully, some indications of liow ue might develop reasonable teaching strategies 
in this area may be found in the data collected in the following sections of the 

lesson material. 

Usage 

A list of words is displayed on the scope. Definitions are given auditorally 
one at a time and the student must Identify the word that matches each definition 
by a touch-probe response. The definitions are chosen with the following 




constraints : 
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1) If the word appears in the Rainbow Dictionary (Wrigjit, 1959) > "the 
meanings defined in that dictionary must be used. 

2) If the word does not appear in the Rainbow Dictionary but does appear 
in the Thorndike -Bamhardt Beginners' Dictionary (196U), at least one 
of the definitions must be used. The choice of usages is governed by 
the criteria of frequency and usefulness in story material. 

3) If the word does not appear in either dictionary, it will not be 
included in the usage section nor used in succeeding lesson materials 
(e.g., sentences, stories). 

A strict "dictionary definition" foimat is avoided in defining word items. 
Standard definitions Lave been re-constructed to stress functional meanings. 

This re-construction can be seen in the following exaii5»le. For the word bat 
one dictionary definition is : "a stout wooden stick or club, used to hit the 
ball in baseball, cricket, etc." In the lesson materials this is reformulated 
in the following instruction; "Touch and say the word that means something you 
might use to hit a baseball." 

A correct response is reinforced by confiixiation of the functional defini- 
tion, "Yes, you hit a baseball with a bat." An incorrecx response is corrected 
in a straightforward manner, "No, you hit a baseball witn a bat. Touch and say 
bat." Thus, for an initial correct response the definition is associated with 
the word twice. For an initial incorrect response, the definition is associated 
with the word fnree times since the student will receive the reinforcing con- 
firmation after his correct response has been made. No branching or optimization 
routines are employed in the usage section. 
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Form Class . 

As part of the concept of "comprehension" of a sentence, one must consider 
the child’s basic understanding of English syntax. One behavioral manifestation 
of a child’s syntactic sophistication is his ability to functionally group vords 
into common form classes (nouns, verbs, modifiers, etc). 'This section provides 
•lesson materials that both assess and teach the form class characteristics for 

the words just presented in the matrix section. 

The following type of problem is presented to the student: 



Dan saw the 


tan 

fat 


hat. 




man 

run 





Only one of the words in the column 
will make sense in the sentence. 
Touch and say the word that belongs 
in the sentence. 

CA: Yes, Dan saw the tan hat. 

Do the next one. 
iCA 

WA: No, tan is the word that makes 

sense. Dan saw the tan hat. 
Touch and say tan. 



The sentence Is composed of words that are in the well-practiced reading vocab- 
ulaiy of the student (i.e. , they have been presented in previous or current 
lessons). The elements of the multiple choice set Include a word which is of 
the correct form class but is semantically inappropriate, two words that are 
of the wrong form class, and of course the correct word. A controlled variety 
of sentence types is employed and the answer sets are distributed over all 

syntactic slots within each sentence type. 

Response choices are categorized in rather broad terns as nouns, v^, 
modifiers, and o:her. The response data are examined for systematic errors 





39 . 




over fl large number of items. Examples of the kinds of questions that will b? 
asked of this data are: 

1) Are errors for various form classes in various sentence positions 
similarly distributed for a given student? for the general population? 

2 ) How are response latencies affected by the syntactic and serial position 
of the response set within the sentence? by the sentence? by the sentence 

length? 

Answers to these questions and others of a similarly general nature should provide 
information that will permit more systematic study of the relationship of sentence 
structure to reading instruction. 

Inquiries . 

Individual words in sentences constitute unique and conversationally correct 
answers to questions. These questions take the interrogative form "Who - - -?, 

Vhat ?, How ?" etc. The ability to select the word in a sentence that 

■miiqugly answers one of these questions demonstrates one form of reading compre- 
hension at the level of the sentence. The set of exercises described in this 
section constitutes an assessment of this reading comprehension ability. 

John hit the ball.l Touch and say the word that 

! answers the question. 

HR 1 Who hit the ball? 

CA: Yes, the word "John" tells 

us who hit the ball. 
iCA 

WA: No, John tells us who hit the 

ball. Touch and say John. 

RR 2 What did John hit? 

CA: Yes, the word "ball" tell us 

what John hit. 

ICA 

WA: No, ball tells us what John hit. 

Touch and say ball. 



O 

ERIC 
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As in the form class section, each sentence Is composed of words from the 
student's reading vocalralary. A wide variety of sentence structures are utilized 
with simple subject-verb-object sentences and progressing to structures 
of increasing complexity. The data will be examined for effects caused by the 
different sentence types. 



Stories Read to the Children . 

From the very beginning lesson the children liave an opportunity to listen 
and react to children’s literature. Thl.s section of the lesson material has 

been designed for the following purposes; 

1) to expand and widen the children's 3iterary experience, 

2) to indicate how the rhythm and flow of language is consistent with 

the thematic material, 

3) to improve the children's listening comprehension in- order to compar 
it in future points with the children's reading comprehension, 

li) to receive the children's reactions to such variables as 

a) thematic contenc of the stor', 

b) length, 

c) style (narrative versus expository form). 



In the actual lesson format, the story is presented un the CRT, sentence 
by sentence, so the child can follow the print at It is read to Mm. Pictures 
are presented with stories if the Project staff telievcc the cMldren need help 
In understanding certain concepts being presented . The mode of presentation 
simulates, in some sense, a parent holding the child in his lap and reading a 
story aloud - an experience which unfortunately many ohildren have missed. 
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At the completion of each story, the following types of questions are asked: 

1) subjective ratings as to positive or negative reactions, 

2) specific questions on factual identification of characters or events 
in the story, 

3) identification of the main theme of the story, 

4) identification of the sequencing of events. 

In a given story only some of these questions are employed; we, in essence, 
let the content of the story determine the type of questions asked. The number 
of questions also varies although typically there are approximately five questions 
per story. 

Stories the Children Read . 

When the children have mastered a sufficient amount of vocabulary, including 
function and syntax words, stories which are actually read by the children are 
introduced into the reading lessons. These original children's stories are 
developed by staff members who are skilled writers and understand children's 
literature. The principal problem encountered in this process is that of creating 
stories around a highly restricted vocabulary. Therefore the following criteria 
were utilized in preparing and editing these original stories: 

1) the variety of literary themes, 

2) the children's familiarity with the thematic content and characters, 

3) logical sequence of events, 

4) simplicity of plot and characterization 

5) appropriateness in the use of the restricted vocabulary, 

6) story length. 

These stories are first inserted in the lessons at Level IV. 



42 . 



The stories are presented on the scope in the sane manner as a story Is 
presented in a story book. The child is to read the story by himself. If he 
does not know a word, he is directed to touch the word and the entire sentence 
in which the word appears is read to him. Illustrations are used with the 
stories as they are needed at certain points to help develop an understanding 
. of the concepts being presented. These illustrations were specified by the 

authors and prepared by the Project's art staff. 

After the child has completed the story, he is presented with a series o 

questions to deteimlne his level of comprehension. The types of questions an 

divided into four main categories; 

1) questions that deal with the direct recall of facta such as the 

identification of characters and the sequence of events, 

2) generalizations about main ideas which relate both characters and 
events within the story in which these characters and/or events are 
not presented in close contiguity, 

3) inferential questions which require the child to relate infoimation 
presented in the story to infoimation stored in his memory about 

his own experience , . . - 

4) subjective questions which include personal ratings and opinions 

of the stories. 

The kinds of questions asked about each story are deteimlned by its content, 
therefore not all stories have exactly the same type of questions. Question 
types are carefully sequenced throughout the stories to provide equal representation 
The number of questions asked about each story varies from 3 to 6 vith generally 
4 questions used. Again the questions and the related stories represent an attempt 
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by the Project staff to collect appropriate information which will help us in 
foiTOilating hypotheses about reading comprehension. Any theoretical development 
in this area must of necessity relate both the child's immediate information 
gathering ability and his ability to logically relate tMs new information with 
concepts stored in his memory system* 

3. Games end Other Motivational Devices 

Rhymes . 

Rhymes are sequenced into a lesson as a listening activity to help the child 
develop competency in the discrimination of the rhyming and alliterative so\mds 
of words and to demonstrate to the child the rhythmic use of language. The 
selection of rl^mies for each lesson is based upon the sound patterns found in 
the matrix section of the lesson. 

Games . 

Games are sequenced into each lesson primarily to encourage continued atten- 
tion to the lesson materials. The games are similar to those played in the 
classroom and utilize the terminology common to game-like situations such as 
baseball or bingo. In addition, each game has been structured to reveal any 
developing linguistic competency on the part of the child. For example, one 
game is centered on the child’s ability to identify a given string of letters 
when presented in a concept-identification paradigm. The question of how children 
come to see orthographic commonalities is relatively unexplored, and the game 
is intended to provide us with some basic information with which 'Je can more pre- 
cisely direct our thinking. Moreover, the games give the children added practice 
on the sequenced vocabulary. 






As indicated earlier, all of these games are intended to provide the children 
with Intrinsic motivation to continue practicing much of the same material that 
has previously teen presented. OrdinarUy the words found in the games were 
introduced In the preceding lesson. Again It Is to be noted that the responses 
in the game sections can be analysed >st as palred-assoclate Items or as concept- 
■ Identification items In more controUed psychological experimentation. 

k, Revlev Lessons 

Two types of review are to be found in the reading lessons. A continuous 
review is inherent in the learning materials which are sequentially Introduced 
In a given lesson and subsequently Incorporated In the activities of succeeding 
lessons. For example, words vdiich are introduced in list and matrix exercises 
In Lesson K are reviewed in the sentence initiator and story materials of Lesson 
H+1. A more conventional type of review is also furnished by lessons designated 
as review lessons. These appear approximately even' seventh lesson. Vocabulary 
and concepts previously Introduced are re-presented but In different formats. 

The nature of the review lessons varies as the vocabulary and skills of the 
student progressively develop. In Level I the review lessons consist primarily 
of straightforward matrix review) that Is, the words which have been Introduced 
in the preceding five to seven matrices are reordered and presented In new 
matrix formats. The review lessons of Level II focus primarily on the recently 
acquired vocabulary items, prepositions, and the inflectional concepts of the 

plural-s, third person singular-s and the -ing suffix. 

In Level III the range of activities in the review lessons is considerably 

furnishes the format for review lessons in the succeeding 



increased and basically 



levels. £::erclses on phonetic discrimination, word meaning, foim class, En g li sh 
word order, rhyming exercises, analysis and synthesis of woris, picture-sentence 
con^refaenslon, compound and polysyllabic words, and verb foxms are Included. 

proiugriON 

Lesson Writing . 

OverftTT Design . An overall design (scone and sequence) for the decoding 
aspect of the curriculum was developed during the first year of the project based 
on the psycholinguistlc propositions stated above. The primary responsibility 
for the overall design was assumed by the principle Investigator, the senior 
research associate and the staff linguist. It must be en^haslzed, however, that 
at every stage In the development of the curriculum the intuitions , experience 
and ejq^rtlse of the entire staff have played a major ro^. 

A well defined sequence of presentation of linguistic patterns is a necessary 
piellmlnaxy to actual lesson writing. The development of such a sequence, however. 
Is a dynamic process which Involves a coiqplex Interaction of study, deliberation, 
eiqperlmentatlon, and evaluation. The sequence as seen In Table 1 was not conqpleted 
before production of lesson materials for the early levels began, nor Is It to be 
considered complete In Its present fojm. The specification of patterns beyond 
Level vm Is a continuing aspect of the current curriculum development effort. 

Wlthln-Level Sequencing . Since the sounding matrix deteimlnes the mono- 
syllabic patterns, the vocabulary available for each lesson and the number of 
lessons to be contained In each level, the next step In lesson writing, after the 
overall- sequence has been established. Is to generate and sequence the sounding 
matrices within a level. The responsibility for this task Is usually assumed by 
a single writer although In the larger levels two writers may work as a team. 
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AU possible words and nonsense words are generated for the level by per- 
mtlng the consonants and consonant clusters about vowels under the constraint 
of the patterns and letters assigned to a given level. The resulting set of 
«,rds and nonsense words are then sorted Into eatrlces ly a trial and error 
n^thod. The goal IS to produce an optical set of nurtrices which give adequate 
•practice for all the patterns. Each matrir must conform to the following 

specifications: 

1) Bo matrix may exceed a size of 26 cells. 

2 ) HO matrix may contain a ratio of nonsense to real words In excess of U0>. 

3) Socially unacceptable words must be excluded. 

I) nonsense words rfilch are misspellings of comaon monosyllabic words 
must be excluded unless they form highly productive syllablas In 

polysyllabic words. 

5) Bbn-English nonsense words are excluded. 

a 4. +hp natter ' has been made in levels I and II. la 

An exhaustive treatment of tne paxxer - 

the later levels, however, the goal has been to provide adequate exposure and 

practice for each pattern and to withhold an Increasing proportion of the possible 

++or.n frvm direct teaching in order to use them as transfer 
words for a given pattern from direcx xeatuxins 

items. 

After the matrix generating writer or team has produced a reasonabl 
Of matrices for a level, the result Is reviewed by the entire staff and sugges- 
tions are made for Improving the sequence. The matrix sequencing process may 
go through several Iterations of trial and review before a consensus Is reached 

and a set of matrices is approved for a level. 



Lesson Level . Once the sounding matrices have been generated and sequenced 
for a given level the actual lesson writing begins. Each lesson which will con- 
tain a matrix is called a "standard lesson," and has many learning tasks and 
problem types. A discussion of the various standard problem blocks will be found 
in the section headed "Curriculum Description." Each writer is responsible for 
the production of some set of problem blocks within a level. A balance is soup.tt 
between the more mechanical blocks (e.g., matrix construction) and the more 
creative blocks (e.g., usage, form class, poems) for each writer's assignment. 

Writing a lesson is quite different than writing a portion of an ordinary 
text book. It is more akin, perhaps, to writing a movie or TV script. Every- 
thing the student is to hear, see, and do mst be clearly defined for every minute 
he is at the terminal. Each display on the CRT must be unambigously specified 
and desired changes in the display must be cued precisely with the audio messages. 
Displays on the projector must be sketched in sufficient detail for co««.unlcation 
with the artists. The timing of projector displays must also be clearly specified. 

Although the branching logic, test criteria, optimization routines and within- 
lesson task sequence have been determined in general by staff consensus, the lesson 
writer must communicate all such information clearly to the curriculum programmers 
for each problem in every lesson. Lesson production forms have been developed 
for all the standard problem types to minimize the amount of repetitive detail that 
is required of the writers and to Insure uniform presen'iation of many of the learn- 
ing tasks for experimental purposes. 

Non-standard lessons are those which present non-recurring or very infrequent 
learning tasks. The lesson which covers the comparative morphemes -er and -est 
is an exam.ple of a non-standard lesson. In all such material, whether it is an 
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entiue lesson or a section of an otherwise standard lesson, the writer has to 
explicate all of the information for each problem without the benefit of forms — 
a tedious time consuming task. 

lbs writing is coursed mainly of experienced primary teachers and 

graduate students in education. It is difficult, however, to define clearly 
what is meant hy the tern "writing staff" since most of the writers have other 
lesponsihlllties within the project and nearly all memhers of the project staff, 
with the exception of the programmers and artists, have made major contributions 
to the writing effort at one time or another. It is even more difficult to 
make a reasonable estimate of the man-hours required to write a lesson. The 
production rate for lesson material. Including editing but excluding audio re- 
cording, art production and programnlng is on the order of 100 lessons a year 
which represents something in excess of 90 man-hours per lesson. 

Editing . Editing the lesson materials is a vital and sensitive process in 
the production of the Stanford CAI Beading Curriculum, fecabulary constraints 

a by the careful sequencing of patterns must not be -/lolated and the 

branching logic of each problem block must be checked for consistency. The 
lesson editor is also responsible for the overaU quaUty of the lesson material 
in the sense of the interest and appropriateness of the sentences, poems and 

stories. 

Several methods of organising the editing process were tried during the 
early phases of the project, such as section sub-editing, and round-roMn multiple 
editing. It was largely through these efforts to control and make manageable 
the extremely cosiplex editorial requirements that the standardised formats were 
developed. The problem of vocabulary control became too difficult in the 
levels to be met by the normal editorial procedures. A dictionary or word Ust 




m 
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was con5>iled which could be stored in a computer and continuously updated. A 
dictionary look-up program was written which matches the displayed text of eacii 
lesson against the stored dictionary and prints out the words used ir* the lesson 
not found in the dictionary up to that point. The resulting print-out is ex- 
amined by the lesson editor to identify those words which are allowable transfer 
. items. Corrections are then made in the lesson text which eliminate the remaining 
words detected by the computer program (i.e., those words not jn the dictionarj*^ 
at that point in the pattern sequence and not usable under the rules for transfer 
items). The above procedure assures a high degree of consistency and accuracy 
in vocabulary control. 

Audio Production . 

Mess^e W'^beriy^c . The first step in producing audio tapes is the numoering 
of the individual audio messages. The coders number each message in the lesson 
books after the lessons have received their final editing. The numbering of 
audio messages is necessary for two reasons. First, the coders refer to the 
messages only by number when coding the lessons. Secondly, the messages do not 
coincide completely with the response requests. As an example of the latter, 
the wrong answer message, as it appears in a lesson consists of two parts: the 

word "no" and a correction message. In the lesson coding this is broken into 
two messages. The second message is played after an overtime response since it 
is not appropriate to say "no" to a student \dio has done nothing. Both messages 

are played after a response which was in error. 

As the messages are numbered, an indicator [X] is placed at each point in 
the message where a display change on the projector or CRT must be made in 
synchronization with the audio. These display changes most commonly invoJve tne 
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addition or deletion of an arrow or an underUne to an existing display, hence 

the X's are referred to as emphasis indicators. 

The messages are nuntered in blocks of fifty to faciUtate future editing 
or revisions. The message code number consists of an alphabetic character 
followed by two digits. The numbering scheme can uniquely identify 1300 mes- 
sages for each lesson. In practice, the average lesson contains approximately 

messages. 

Script Preparation . After the messages have been numbered hy the coders, 
an audio script for each lesson is produced ly the typists from the lesson books. 
The audio script is necessary for speed and accuracy in recording. All irrele- 
vant material in tbe lesson book, such as response identifiers and branching 
instructions, is ignored in the production of the audio script. The messages 
are typed in sequential order. The only material Included in the audio script, 
besides the actual messages, are the message numbers and the en^hasls indicators. 

The scripts are carefully proofread for accuracy. Minor typographical 
errors such as substituting ^ for f^ and misplaced emphasis Indicators are 
difficult to detect except by team-proofing against the original lesson material. 
Such errors, if missed in the proofreading, are increasingly more difficult to 
correct at each succeeding step in the audio production processes. 

Recording . The master narration tape is recorded on an Ampex PB-10 two- 
track tape recorder. The messages are recorded on the "A" channel using an 
IflV 1800 cardioid microphone. A constant 400 cycle tone is recorded on the "B" 
channel in conjunction with each message from a Hewlett-Packard 200 AB oscillator 

controlled by a tone key. 

The requirements of the system, which wiU be discussed in detail in the 
section "Audio Compiler," specify that the UOO cycle tone must be generated hy 
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depressing the key at least 0.25 seconds before the message is started and con- 
tinued until at least 0.25 seconds after the message is finished. Further, the 
intexmessage gsp must be at least 3 seconds and less than 10 seconds in duration 
and the emphasis indicator must be signaled l)y a cessation of the tone of at 
least 0.5 seconds and no more than 1.5 seconds. 

Practical considerations of audio consumption at the student terminals re- 
quire that the above minimum specifications be adhered to as nearly as possible. 
The recording narrators must not only exhibit pleasant voice quality, clear 
enunciation and good intonation patterns but also develop considerable skill in 
the use of the tone key. 

The master narration tapes are produced in a sound-proof recording studio 
at Ventura Hall on the Stanford campus by two female narrators, each recording 
for four hours at a session. Experience has shown that a narrator cannot 
maintain accuracy and efficiency over longer periods. 

Each lesson is concluded with a story which is read to the student while 
the orthography is displayed on the CRT. This section of the lesson, known as 
the "listening stories" is recorded by a group of professional actors from the 
Stanford Repertory Theater. Professional actors are utilized for the recording 
of this material to insure an interesting, dramatic, and appropriate presentation 
of the stories. It ?s assumed that such a presentation will maximize the motiva- 
tional aspects of this section of the lesson material and at the same time provide 

an effective introduction to literature. 

Since the orthography is displayed sentence by sentence, each sentence is 
considered as a separate message and no emphasis indicators are necessary to 
synchronize the CRT display and the audio. The actors then are only required 
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to make the appropriate tone key manipulations betveen sentences, a technique 
vhich is quickl^sr learned, and can therefore concentrate their attention on 
producing an artistic reading. 

The master narration tape for each lesson is edited and corrected by the 
narrator Innediately after recording.' The tapes are then traded and given a 
final edit with each of the tvo narrators editing her colleague’s tape. The 
major aspects of the narration tape which are carefully attended to in the 
editing process are: correctness of the messages, clarity of enunciation, proper 
tone signal overlap, and accuracy of the e»®hasis indicators. The rather elabo- 
rate and painstaking editorial precedure is a vital part of audio production 
since correction of tapes is very difficult once they have undergone the audio 

assembly piocess. 

Ascemblv . After a master narration tape has received the final edit- 
ing by the narrators it is sent to the Brentwood laboratory for the generation 
of a master machine tape. A machine master is a four-track tape. The first 
two tracks contain the audio messages taken from the master narration tape. The 
third track is reserved for on-line student recording and the fourth track con- 
tains the tape segment addresses. 

A machine master is produced by a computer program which reads the tone 
signals on the master narration tape, e-»-aluates the length of a set of such 
signals and records their associated verbal messages on the machine master using 
an optimal packing strategy. Figure 11 is a graphic example of what is meant 

by "optimal packing." 



Insert Figure 11 about here 









Audio Assembly Routine 
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verbal messages of varying length are recorded on the two message tracks 
in a maimer which maicimizes the nnmber of messages per unit length of tape and 
^nlMzes the distance between any two messages. The latter condition also 

minimizes the search time required for positioning any message. 

The result of the optimal packing strategy is a non-sequentlal ordering of 
.messages on the machine master. To control an otherwise chaotic condition,, a 
table is generated concurrently with tte recording of the messages which specifies 
the location on the machine master 0^ track and se^ent number) of the beginning 
of each message, its sequential number on tte master narration tape and its 

length. This table, called the 

disk for each lesson. A printed audio symbol table is also produced for use by 

the programmers in the debugging process. 

The complete process of producing a master machine tape and the associated 

tables is known as audio assembly and is carried out during production hours 
by a single computer operator, typically between midnight and 8 A.M. 

Taue Buolication . The terminal tapes are produced from the master machine 
tape on a Viking 235 IXipUeator. Three duplicates can be made simultaneously 
on the Viking at 7 3/2 inches P®*' second. T^e duplication is under the control 
of a single technician and the duplication time follows an intermittent demand 

schedule since the process is Independent of the 1500 system. 

The terminal tape requirements for each day are letermined by the dlstrlbu- 
tion of students over the lesson material. After all students have completed a 
given lesson, all but three terminal tapes for that lesson are degaussed. The 
three remaining duplicates and the machine master are placed in storage. Three 
duplicates have been found to be sufficient to act as a buffer to absorb new 
Students and for demonstration purposes- 
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Art Production . Production of the art wik for the reading curriculum 
begins vlth the lesson witers uho, in the course of writing the lessons, specify 
the necessary iUustretions. The art specifications are made on the "projector" 
section of the visual pages of the lesscM by means of rough sketches and descrip- 
tive notes. Illustrations are required for the word list sections, seme sentence 
analysis sections, and all of the listening and reading stories and poems. 

The amount of detail In the specifications necessary for communication betwee. 
writer and artist varys widely for the different sec >ns. For exan5.1e, in the 
uord-ust section each illustration is weU defined hy the individual words and 
their sentence context. A minimum of specification is therefore required in 
the word-list section. On the other hand, the salient points of the story section 
Uhich the author wishes to «!®hasize in each Illustration must be clearly defined 

for tUe artists. 

Aa in aU phases of lesson production, the commmlcatlon channels between 
the two groups, writers and artists, are open for a continuous two way flow of 
info-vatlon. The illustration requests of the writers must occasionally be 
adjusted in terms of the artist’s abiUty to visually convey abstract terms and 
the practical limitations of time and production facilities. Such ccm5.romlses 
often require rewriting of individual frames. The early stages of art prepara- 
tion for a block of lessons is therefore characterized ty an open dialogue 
between, the writers and the artists. 

Art production requirements are currently being met by an art editor, one 
half-time and three full-time artists. The size of the art staff is determined 
hy a coa^waiise between long range requirements for Illustrations and the space 
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available to bouse the artists. In order to :«et our total art co-lt»ents 
the present staff Is required to produce an average of tuenty plates a seek per 

artist. 

^techniques and iredla are dictated by the demands of high production and 
Mgh quality. »,st of the plates are done vith felt tip marher on tuo-ply 
. .rtstol board, vlth occasional use of colored Inh, challr, paste-up and tempera. 

M additional factor in the choice of «dla and nalette is the ccm^lex set of 

requirements of color plaoto-reprcductlon. 

A serious attempt has been made to include In the pictorial iUustrations 

a balanced distribution of racial an ethnic figures as a reflection of the 
of those elements in Ibaerican society. The goal has been to feature 
aU racial and ethnic t;.qes as central figures engaged in interesting activities, 
by means of artistically and socially ho cst and appropriate drawing. 

Photo Reproduction . 

The illustrations are photographed in blocks of lessons vdilch are defined 
by the ^ structure of the overall sequence. The shooting is done at the 
ft,apital Laboratory of the Stanford Shoto Service with a l6 ™ «,vie camera and 

single frame exposure. 

correct sequence of illustrations is vital, therefore the art editor, who 
originally assembled the plates, is present to assist the photographer at every 
Shooting session. The plates are shot in blocks of lessons with 9 blank fra.es 
left between blocks to minimise the difficulty of spUcing if later corrections 

or additions are necessary. 

inimratorv which Is examined by 
A work print Is produced by the processing laboratory wux 

m.. V -.-IM- Is stenned throu^ an IIM '512 projector at the 
the art editor. The work print Is steppea tnivutiix 

and is checked for proper sequence and acceptable color. 



Brentwood Laboratory 
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When a no* print has recei«d the final approval of the art editor it 
is returned to the processing laboratory and a release print is made. Tie 
release print, which contains pictures hut no identification code, is cosibined, 
through a process hnown as A-B printing, with a code irester to produce a master 
nim. The master film is then used to produce the sire terminal prints for de- 

• bugging on the 1500 system. 

The Sim terminal prints are mounted on self-threading, dust proof cartridges 
and are used, along with the audio tapes, in debugging coded lessons. When the 
debugging process is completed, a quantity of terminal prints are produced and 
mounted which will assure sufficient films and reserves to accamnodate the 

number of students anticipated to be working in a level at any given time. 



Coding . 

Before the lessons can be used in the CAI system they must be translated 
into a language which is understood by the coeqiuter. The first step in this 
process is acconrllshed by tbe lesson progranmers by coding the lessons in 
Coursewriter II — a CAT source language developed by IW. * eo^ed lesson is 
actually a series of Coursewriter II coemiands which cause the cosqjuter to display 
and manipulate text and graphics on the CRT, position and display film in the 
projector, position and play audio messages, accept and evaluate keyboard and 
li^t pen respor.ses, update the performance record of each student and liiq.lement 
the branching logic of the lesson flow by means of manipulating and referencing 
a set of switches and counters. A typical lesson wiU require 9,000 Coursewriter 

II comnands for i‘*'s execution. 

Table 3 shows ^he audio messages and film Images required for two sample 
problems along with possible addresses on the audio tape and film strip. What 
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Audio Script and Film Imagps vith ^rpothetical Addresses 




AOl: Tbuch say the word that goes with the picture. 

A02: Good. Bag. Do the next one. 



A03: No. 

AOU: The word that goes with the picture is bag. Touch and 



say bag. 

A05: Good. Card. Do the next one. 

A06: Mb* 

AO?: The word that goes with the picture is card. Tbuch and 

say card. 




Film str i; 

Address Picture 
FOl: Picture of a bag. 

P02: Picture of a card. 
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foUows are the comp-iter conmands required to present two problems described 
in 3, analyse the students' responses, and record them on the data record. 

The left column lists actual commands to the computer contioUing the instruction, 
(Labels LI, L2, etc. in the column on the far left indicate points which can be 
branched to.) On the right is an explanation of the results of the execution of 
these commands The first problem is explained command by command; the second 
problem is explained only in outline. 

Insert Table 3 about here 



Commands 

Pr 

LD 0/Sl 

FP FOl 
DT 5,l8/bat/ 

DT 'J,lS/\>ag/ 
DT 9,l8/rat/ 
AUP AOl 

LI EP 30/ABCDl 



Explanation 

Problem: Prepares machine for beginning of new problem. 

Load: Loads 0 into the error switch (Sl). The role of 

switches and counters will be explained later. 

Film Position: Displays frame FOl (picture of a bag). 

Display Text: Displays "bat" on line 5 starting in 

column l8 on the CRT. 

Displa^rs "bag" on line 7 starting in column l8 on the CRT 

Displf.ys "rat" on line 9 starting in column iB on the CRT 

Audio Play: Plays audio message #A01. "Touch and say 

the word that goes with the picture. 

Enter and Process: Activates the light pen: specifies 

the time limit (30 seconds) and the problem identifier 
(ABDCl) that will be placed in the data record along 

to this problem. If a response is 



with all responses 



Commands 



AD 1/Ch 
LD 1/Sl 
AUP AOU 

ET 7,i6/V 

BR LI 



CA 1,7, 3, 18 /Cl 



Explanation 

made within the time limit the computer skips from 
this command down to the CA (correct answer comparison) 
command. If no response is made within the time limit, 
the commands immediately following the EP command are 
executed. 

Add: Adds 1 to the overtime counter (C4). 

Load: Loads 1 into the error switch (Sl). 

Audio Play: Plays message #A04. "The word that goes 

with the picture is bag. Touch and say bag." 

Display Text: Displays "^" on line 7, column 16 (arrow 

pointing at "bag"). 

Branch: Branches to command labeled LI. The computer 

will now do that command (EP) and continue frcan that 
point. 

Correct Answer: Compares student’s response with the 

area 1 line high starting on line 7 , 3 columns wide 
starting in column 18 on the CRT. If his response 
falls within this area, it will be recorded in the data 
with the answer identifier Cl. When a correct answer 
has been made, the commands from here down to WA 
(wrong answer comparison) are executed. Then the 
machine jumps ahead to the next PR. If the response 
does not fall in the correct area, the machine skips 
from this command down to the WA command. 
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Command Explanation 



BR L2/S1/1 


Branch: Branches to command labeled L2 if the error 

svitch (Sl) is equal to 1. 


AD 1/Cl 


Add: Adds 1 to the initial correct ansver counter (Cl). 


][i2 AUP A02 


Audio Play: Plays audio message ^02. "Good. Bag. 

Do the next one . " 


WA i,5,3,i8/wi 


Wrong Answer: These two commands compare the student 


WA 1,9,3, 18/W2 


response with the areas of the two wrong answers, that 
is, the area 1 line high starting on line 5, 3 colums 
wide starting in column l8, and the area 1 line high 
starting on line 9, 3 columns wide starting in columxi 
l8. If the response falls within one of these two 
areas, it will be recorded with the appropriate identifier 
(Wl or W2). When a defined wrong answer has been made, 
the commands from here down to UU (undefined answer) are 
executed. Then the computer goes back to thr EP for 
this problem. If the response does not fall in one of 
the defined wrong answer areas, the machine skips from 


AD 1/C2 


this command down to the UK command. 

Add: Adds 1 to the defined wrong answer counter (C2). 


L3 LD 1/Sl 


Load: Loads 1 into the error switch (SI). 


AUP A03 


Audio Play: Plays message ^03* No. 


AUP A04 


Audio Play: Plays message -ifAOh, "The word that goes 

with the picture is 'bag'. Touch and say bag." 


DT 7, l6/->/ 


Display Itext: Displays on line 7, column l6. 
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Commands 

UN 



AD 0/C3 
BR L3 



Explanation 

Undefined Wrong Answer: If machine reaches this point 

in the program, the student has made neither a correct 

nor a defined wrong answer. 

Add: Adds 1 to the undefined answer counter (C3). 

Branch: Branches to command labeled L3» (The same 

thing should be done for both UN and WA answers. This 



PR 

LD 0/Sl 
FP F02 

nr 5,18/card/ S 
nr 7,18/cart/ 
nr 9,18/hard/ 

L4 EP 30/ABCD2 
AD 1/Ck 
LD 3/Sl 

AUP A07 ) 

DT 5,i6/V 
BR LU 



branch saves repeating the commands from L3 down to UN. ) 
Prepares the machine for next (2nd) problem. 

These commands prepare the display for the 2nd problem. 
Notice the new film position and new words displayed. 
The student was told to "do the next one* when he 
finished the last problem so he needs no audio message 

to begin this. 

Light pen is activated. 

These commands are donr only if no response is made in 
the time limit of 30 seconds. Otherwise the machine 
skips to the CA command. 



CA 1,5,U,18/C2 Compares response with correct answer area 
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Commands 

BR L5/S0/1 
AD 2/Cl 
L5 AUP A05 



Explanation 

"I 1 is added to the initial correct answer counter unless 
> the error switch (Sl) shows that an error has been 
made for this problem. The student is told he is 
correct and goes on to the next problem. These 



WA l,'J,k,lQA^3 
WA 1,9,^A8/W^ 
AD 2/C2 
L6 LD 2/Sl 
AUP A06 
AUP AOT 

ET 5,16/-^/ 




> 






UN 



commands are executed only if a correct answer has 
been made. 

Compare response with defined wrong answer. 

1 is added to the defined wrong answer area and the error 
switch (Sl) is loaded with 1 to show that an error 
has been made on this problem. The student is told he 
is wrong and shown the correct answer and asked to touch 
it. These commands are executed only if a defined wrong 
answer has been made. 

An undefined response has been made If the machine reaches 



AD 2/C3 
BR L6 




this command. 

is added to the undefined answer counter and we branch 
up to give the same audio, etc. as Is given for the 



defined wrong answer. 



Thirty counters that can be used to keep track of a student's perfomnance 
are available to the lesson programmer. During the instructional flow the current 
values of these counters are used to make branching decisions regarding what 
stimulus materials are to be presented next. For example, if the correct-answer 
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counter for a particular class of problems has a high value, the student may be 
branched ahead to more difficult topics, whereas for a low value he may be branched 
to remedial work. These counters can contain any number from 0 to 32,767- They 
are set at zero at the beginning of a course and added to when desired. 



For example, counter k (CU) was used 'to record overtimes; each time the time 
■ limit was exceeded one was added to counter U (AD l/CU). 



There are also 32 switches available to the instructor. A switch Is either 
in the zero or one position. These are used to keep track of previous events. 
For example, at the beginning of a problem, zero Is loaded into SI (the "error" 
switch) . This indicates that no error has yet been made on this problem. If 
the student makes an error on the problem, one Is loaded Into SI. Then, if a 
correct answer Is made on his second try, the command can be branched around 
adding one to the Initial correct answer counter because the error switch (Si) 

is equal to one. 



There are many features of the CAI system that are not demonstrated by the 
simplified example presented here. The pattern of the problems may vary- widely 
from this sample. At various points In a lesson, criteria may be set which, 
if not met, may cause the student to branch to remedial problems or have the 
proctor called. Parts of the CRT display may be underlined or displayed In 

synchronization with the audio messages. 

While a student is on the system he may comple-e as many as 5 or 10 problems 

of the type shown above per minute; providing a significant amount of coded 
lesson material for student use Is a major problem. The typical procedure in 
the reading progr»; is to present material in blocks such that the problems are 
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alike in fomnt, differing only in certain specified ways. The tvo example 
problems differ only in l) fll» display, 2) vords displayed, 3) problem 
identifier, k) the three mrdio nmbers, 5) rov display of (correct answer 

row), 6) correct answer area, T) correct answer identifier. This string of 
code can be defined once, given a two-letter name, and used later by giving a 
one-line macro coavaand, the specifics which vary from problem to problem are 

called parameters. 

The use of macros cuts down greatly the effort required to present many 
different but basically similar problems. Por example, the two problems which 
were presented command by command above would be presented in macro format. 

Problem 1; CM PW]F0l]bat]bag]rat]A0l]ABCDl]A0h]A02]A03]7ll»7»3,l8]Cl] 

Problem 2: CM PWlF02]card]cart]hard]]ABCD2]A07lA05]A06]5U,5,U,l8]C2] 

The co-and to call a macro is CM and PW 7s our two-character code for the macro 
involving a picture -to-word match. Hotice that in problem 2 there is no intro- 
ductory audio message; the "]]" indicates that this parameter is not to 

filled in. 

The macro capability of the source language has two distinct advantages 
over code written command by command. The first is ease and speed o 
The call of one macro is obviously easier than writing the comparable string of 
code. The second advantage la increase in accuracy. Not only are coding errors 
drastically curtailed, but if the original macro is defective or needs to be 
changed, every occurrence of it in the lesson coding can be corrected by modifying 
the original macro; in general, the code can stay as it is. The more 
the problem format the more valuable the macro capability becomes. Apart from 
a few non-standard instmictlonal audio messages and display items, approximately 
90-951( of all the reading curriculum has been programmed using roughly 110 basic 



macros. 
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The final step in translating the lesson material to a form usable in the 
computer is the lesson assembly process. A series of machine language programs 
read in the coded lessons, expand the macros, translate the audio code to actual 
tape addresses from the audio assembly table and finally read out the assembled 

lesson translated into binary code onto a disk. 

The final editing steps are carried out at the student teminals in a process 

oaued debugging. Each lesson is examined in detail by mahlng all correct and 
all incorrect responses. Errors left uncorrected from any stage in the produc- 
tion process are detected as they might be seen by a student. Corrections are 
made and the lesson is gone through again in a similar manner. This Iterative 
debugging process is vital for the assurance that the student wlU not suffer 
fiem human error inherent in the complex process of lesson production. 

SOME PRELIMINARy RESULTS 

Weekly Report . 

A report on student progress at a fairly gross level is generated each «eek 
primarily for the Information of the teachers. The report contains the individual 
student's name, Identification of the lesson rn which the student is working, 
the number of proctor calls received for that student during the week, end a 
cumulative weighted index of the student's performance for each of the six major 
problem blocks (i.e., letter discrimination, word presentation, matrix construc- 
tion, comprehension, compound words, polysyllabic words). Also included in the 
report are the cumulative number of sessions the student has hsd on the machine, 
his number of absences during the week and the total amount of time off the 
system that week. The last three categories are computed from the teaching 
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proctor records. The remainder of the report is generated hy reading pertinent 
items from the student records. The perfoimance index for each student in each 
of the six problem blocks is computed in the following manner. A series of six 
counters are assigned for the computation of the perfoimance index. As a student 
proceeds through a problem block the counter for that block is indexed to in- 
dicate the proportion of initial correct responses made. When the student has 
camplsted the block the contents of that register are used to update the associated 
performance index in the student record. The current value for the register is 
read out and the new value is added. The updating of the register, however, is 
done using a weighting formula: i-kO) + (.60). The weighting 

procedure is used to reflect more accurately the student's current performance 
in relation to his past performance. Conqputations are done, however, in the 
integer mode which makes the index somewhait conservative. 

Althou^ the weekly performance report is prepared primarily for the class- 
room teacher and the teaching proctors, it provides sufficient data for examining 
several rather general questions of overall student performance. The weekly 
student performance record has been used in the two following analyses. 

Spread on Main Line . A central core of problems within the lesson material 
are considered main line problems in the sense that they are problems which 
each student must master. A student may be branched around blocks of main line 
problems by successfully passing certain screening tests. On the other hand, 
a student may be branched to appropriate remedial material if he has difficulty 
with these central problems; but in every case he is returned to that set of 
main Unp problems for which remedial material was introduced. 
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Each lesson contains an awrage of 125 main line p.'xjblems. Therefore, the 
number of lessons convicted by a student may be used as an index of the number 
.^f main line problems successfully co»®leted. Figure 12 shows the number of 
^ line problems cos^leted each week by the fastest, slowest and median student 
Figure 12 was derlsed by identifying the student who bad coB®leted the most 
lessons on the final student progress report for the week of June lUth and, 
from the same report, identifying the median student and the slowest student, 
considering only those students who had begun the program on the 15th of Bov- 
aaber, 1966. Hew students who had mowed into the school and the remedial second- 
grade students are not considered in the derivation of Figure 12 - or in ar^ 
of the data reported in this paper. 

Insert Figure 12 about here 



The year ended with a difference between the fastest and slowest student 
of U,125 problems completed. The inter-quartlle ranee was 1,375 problems and 
the median student completed 2,625 main line problems. There was, however, a 
rather wide variation in tte asr>unt of time spent on the system by the students 



in order to take this variation into account a rate of progress score was com- 
puted by dividing the number of problems coo 5 . 1 eted at the end of the year by 
the number of sessions that the student had on the system. The cumulative rate 
of progress for the highest, lowest, and median student is shown in Figure 13, 
expressed in terms of number of main line problems co.®leted per hour of instruc 
tlon. The range in rate of progress was between 35 problems per hour for the 
Slowest student to 170 main Une problems per hour for the fastest student. The 



inter- quart ilfi range is U5 to 



110 with the median at 75 problems per hour 



o 




NUMBER OF MAIN-LINE PROBLEMS 




Figure 12. 

Cumulative Number of Main-Line Problems Completed 
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Insert Figure 13 about here 



From the standpoint of both the total number of main line problems cos^ileted 
during the year and the rate of progress. It is clear that the CAI reading cur- 
rlculm is accounting for individual differences on at least one dimension (i.e., 
the movement of the Individual student through the lesson material). The 
differences noted in Figures 12 and 13 Must not be confused with a variation in 
rate of response. The differences in the rate of response between students was 
very small; the average rate was approximately four responses per minute. The 
differences in the total number of main Une problems con?pleted and in the rate 
of progress can be accounted for by the amount of remedial material, the optimiza- 
tion routines, the number of corrections, and the number of accelerations for 
the different students. 

Sex Differences . It has been a common finding In reading studies that girls 
generally surpass boys in the acquisition of reading skills and in reading per- 
foniiance, particularly in the primary grades (Gates, 196I; Wyatt, 1966). The 
suggestion has been made many times that these differences mi^t be attributed, 
at least in part, to the social organization of the classroom and to the value 
and reward structures of the predominantly female primary-grade teachers. It 
has also been argued that because of differences in developmental rates first- 
grade girls are more facile in visual memorization than boys of the same age — 
a facility which would favor the girls In the essentially sight-word method of 
vocabulary acquisition commonly used in the current basal reading series. If 
these two arguments are viable, then one would expect that placing students In 
an asocial environment such as a CAI tutorial system and presenting a linguistically 



68 . 



oriented curriculm which e.®hnsi.es analjrtic skiUs, as opposed to rote i«»ori- 
zation of words, would niniinize the sex difference in reading performance. In 
order to test this notion, the rate-of-progress scores taken from the final 
teachers' report were rank ordered and tested for significant sex effects using 
a Mann-Whitney U-Test. The null hy^thesis in this and in the following tests 
is that the scores for the boys and the scores for the girls haye the same 
distribution. Tte test of sex effects yielded z of .05. Under the nuU hypothesis 
the probability of z being greater than or equal to 0.35 is 0.36. Sex difference 
then is not an influential variable in the rate of progress in the Stanford CAI 

reading curriculum. 

sex differences still might be an influential factor in accuracy of per- 
formance. TO test this notion the final performance index scores for each of 
the four standard problem blocks reported on the weekly teachers' report were 
«nk ordered and examined under the Mann-Whitney U-Test. The results were as 

follows: 

Letter identification; Pr(z > 0.33) = 0*37* 

Word list; Pr(z > I.83) = 0.03. 

Sounding matrix; Pr(z > l.Ul) = O.08. 

Sentence ccxnprehension; Pr(z >1.37) ~ 0.09* 

The only significant difference, at the 0.05 level, was found in the word list 
scores. However, the scores in the matrix and conq-rehension sections were in 
the expected direction (i.e., girls excelling boys). These results, while 
by no means definitive, do lend support to the notion that when students are 
removed from the normal classroom social milieu and placed in the asocial 
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Figure 15. 

Cumulative Rate of Progress of Fastest, Median, and Slowest Students 
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Figure 13- 

CiMulf tlve Rate of Progress of Fastest, Median, and Slowest Students 
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environment of a CAI tutorial system, boys perfom as well as girls in overall 
rate of progress through a reading curriculum. The rosults also tend to support 
the idea that in a CAI environment the sex difference is minimized in direct 
proportion to the en®hasis on analysis versus rote memorization in the learning 
task. The one problem section whexe' the girls achieved significantly higher 
scores than the boys was the word list section which is a paired-associate list 

learning task. 



Analysis of the ResT)onse Data . 

Each student response is first stored in an area of core memory called a 
response buffer. When the limit of the storage capacity of the rosponse buffer 
is reached, the contents of the buffer are read out on a response tape. The 
steps in the preparation of the response tape for storage and for serious analysis 
of the student's response data is illustrated in Figure l4. A co^ressed response 
tape is generated from the data response tapes by means of e compressor routine 
which removes the inter-record gaps on the daily response tape. It al.so may be 
used to compress two or more previously compressed response tapes. It is the 
compressed response tape which is held in permanent storage. 

Insert Figure l4 about here 

There are items in the student response record which are of less than im- 
^diate interest insofar as data analysis is concerned. Therefore, the cor^ressed 
response tape is subjected to a strip routine which strips out those items which 
will be used in immediate analysis to form a subset response tape. In the case 
of the reading pix^gram the items stripped out are the student number, course 
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name, response identifier, the match identifier, and the latency record. The 
strip routine number 1 may also be used to select certain records of interest 
from the compressed response tape. 

A generalized sort/merge program is applied to the subset response tape 
which will handle up to ten key words in any order within a record and which 
preserves chronological o 3 'er in cases where two or more entries are the same. 

^ means of the generalized sort/merge routine a sorted subset tape is generated 
where the records are recorded first according to student number and second 
according to problem number (response identifier). A tape-to-printer routine 
is associated with each tape, from the dail^^ response tapes to the sorted sub- 
set tape. This routine allows for listing of the particular tape which then 
may be examined for con^leteness and accuracy. It is the sorted subset tapes 
which are used as input for data analysis programs. The first data analysis 
program which will be discussed here is in reality a data description program 
at the level of the problem type. 

Due to the complexity of the branching routines within a problem type and 
the variation of the branching logic between problem types, it is possible to 
have an indeterminate number of responses to a given problem by a given student. 
These responses then must be categorized. A reasonable method of categorizing 
such responses is illustrated in Figure 15- 



Insert Figure 15 about here 




COMF1: COMPressor routine number 1. 

DAPxn: Data Analysis Program x number n. 

SORTl: SORT routine number 1. 

STRPl: STRiP routine number 1. 

TTPl: Tape To Printer routine number 1- 

TTP2: Tape To Printer routine number 2. 



Figure 



Data Analysis Programs 
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Figure 15 • 

Response Categorization at the Problem- Oype Level 
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The Initial response to a problem vill fall into cell 1. If the response 
is incorrect, the student will te retired to «s;pond >^ain to the same problem. 
Subsequent responses, until a correct response is made, vill then fail into 
ceU 2. If the problem type uses an optimization routine, then the list of 
p„blems Will be gone through again, deleting those problems which received an 
. initial correct response. On each optimization rnm there will be a first time 
for a student to respond to each problem. Those first responses will fall in 
cell 3. However, as in cell 1, that first response may be in error, in which 
case subsequent responses will be called for on that problem which will then 
fall in cell U. There are many cases in the curriculum where a student will be 
branched out of a problem type into some appropriate remedial material and then 
he returned to the original problem type. The initial response to a problem in 
e subsequent entry to a problem type will fall in cell 5 and corrections in 
cell 6. The initial responses of optimization runs on subsequent entrle 
fall in cell 7 and subsequent correction responses will faU in cell 8. 

This method of classification of responses uniquely defines each response 
by cell number. Civen a sorted subset tape, as specified above, a general 
algorithm was developed to assign a cell number to each response. The general 
algorithm is as follows : For a given student and a given problem n^ber the 

first response encountered is assigned to cell 1. The second to n responses 
are assigned to cell 2 where n equals the first CA match identifier. The n.l 
response is then assigned to cell 3. The following responses with WA match 
identifiers, if any, and the next CA match identifier are assigned to cell U. 
The next response is then assigned to cell 5- -rhe remaining Instnictlons are 
essentially a repetition of the first portion of the algorithm. It must be 



ei^lzed, hoover, that this Is a general algorltta to ahlch several variations 
mst he added to account for the unique branching logic of many of the problea 

types. 

In addition to the above described cell Identification each response Is also 
tagged vlth a restart run nunber. This restart Identification Is required since 
a student uho Is signed off for any reason (e.g., end of the period, machine 
failure) uhlle he Is at soeie point ulthln a problem block can only start again 
at the previous restart point. This situation l.q.Ues that the probaMUty Is 
Mgh that at the beginning of a day a student ulU be repeating s«i« indeterminate 
number of problems from the day before. Those responses uhlch result from re- 
peated e:®osure to problems because of restart runs, must be treated in data 
analysis In a different manner than those responses vhlch sere given to the original 

«®osures to the problems. The association of problems to restart runs Is done 

durlnf the sort routine between the subset response tape and the sorted subset 
tape. 

With each student response tagged with a cell number and a restart number, 
the process of compiling a series of sumaary repc-ts was begun. Several levels 
of su»«ry reports can be generated. Reports can be generated which describe 
the performance of the total class of students at the level of the problem block, 
the problem type, and the Individual problem. Other reports can be generated 
which will sumaarlze performance of Individual students at the three levels, 
block, type, or problem. Tbs first series of reports which are in the process 
of generation at the time of this writing, describe the performance of the total 
student group at the level of the problem type. A separate report is produced 
for each problem type within each lesson. The report describes the mean, standard 
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deviation, and range of the distribution within each of the el^t cells for: the 
number of problems, number of responses, the proportion of responses which were 
Incorrect, the proportion of problems with at least one error, the correct answer 
latencies, the wrong answer latencies, the overall latencies and the CA, WA, 
and overall speed scores. 

Achievement Tests . 

Even though this first year's operation of the system has been viewed as 
essentially an extended debugging process. It was felt that a conqprehenslve 
end-of-tle-year testing program might yield some Interesting Insights to the 
potential Intact of the program on overall reading achievement. Accordingly , 
a battery of tests was assembled which would measure achievement In each of the 
major areas of reading behaviors taught at the first-grade level. The behaviors 
Identified and Included In the testing program were: 

1. Identification of letter and letter- strings. 

2. Acquisition of an Initial sight vocabulary. 

3 . Acquisition of word decoding skills: 

a. Initial consonants 

b. Initial consonant clusters 

c. medial short vowels 

d. final consonants 

U. Acquisition of reading comprehension skills: 

a. word meaning 

b. syntax or form class for sentence comprehension 

c . paragraph comprehension : 

1 . main Idea 



2 . 



recall of facts 



meiever possible ve chose standardised tests of the reading behavior to be 
evaluated. In several cases, the tests only approximated what we were trying 
to measure or were not pure tests of a single reading behavior. As far as we 
could determine, none of the available test batteries included tests of all the 
reading behaviors listed above. It w; > particularly difficult to find group 
administered tests measuring a child's ability to decode words and the child s 
knowledge of syntax or form class behaviors, therefore Ruth Hartley of the 

project staff devised tests of these behaviors. 

The tests which were chosen for this test battery were derived fn® the 

following sources: 

1. Gates-MacGinitie Reading Test, Primary A, Form 1 , 1965 edition. 

2. California Reading Test, Lower Primarj^ Form W, 1957 edition with 
1963 ncras. 

3. Stanford Achievement Test, Reading Sections, Primary ! Battery, 

I96U edition. 

U. Project developed tests. 

The students in the CAI mathematics program provided an ideal control group. 
Analysis of the individually administered Stanford Binet I.Q. tests at the be- 
ginning of the school year indicated that the students in the reading program 
and the students in the mathematics program may be considered as two samples 
from the same population. The mean I.Q. for the reading group was 92-5 (standard 
deviation 15.6), and the mean I.Q. for the mathematics group was 9I.8 (standard 
deviation IU.6). Any "Hawthorne effect" which might be induced by the CAI 
ejcperience is controlled since the mathematics students had an equal amount of 
time on the syscem but for a different subject matter. The mathematics students 
received a program of reading instruction which might be termed traditional, 

the Ginn the Alyne and Bacon first-grade readers. 



relying primarily upon 
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All tests, with the exception of the Stanford Achievement Test, were admin- 
istered by members of the project staff, assisted by graduate students from the 
School of Education at Stanford University. Each testing team, consisting of 
a project staff member and a graduate student, administered a given test for all 
of the students in order to eliminate tester effects. The Stanford Achievement 
-Test was administered by the individual classroom teachers as part of the testing 
program of the State of California. All tests were conducted within the normal 
classroom setting. The testing program was carried out during the month of 
May, 1967. 

The content and characteristics of the three standardized tests in the 
battery are well-known and need not be reiterated here, however, a brief descrip- 
tion of the project developed tests might be helpful in the interpretation of 

the results. 

Vocabulary Test . This test consisted of words and patterns that were 
presented in the CAI program. Some of the words and all of the word patterns 
included in this test also appear in the Ginn and Alyne and Bacon first-grade 
basic readers. The test consisted of 20 sets of three words each. The student 
was directed to circle the word pronoiinced by the examiner. 

Phonetic Discrimination . This test was designed to check the child's ability 
to distinguish between minimal contrasts of monosyllabic words and to connect the 
auditory sound of a phoneme with its grapheme. The test contained two sections: 

1) the student was required to identify in a set of three words the word pronounced 
by the test administrator (e.g., bat, rat, sat) and 2) the student was required to 
fill in the missing letter or letters in a word pronounced by the administrator. 
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In both cases the test dealt with initial consonants, initial consonant clusters, 
medial short vowels and final consonants, 

Reql^^ 1 ^g Comprehension: Understanding of Sjmtax or Form Class Behaviors . The 
twenty items of this test consisted of a series of sentences in which one word 
was omitted. The subject was required to choose, from a set of four words, the 
. word which would complete the sentence. The distractors in the multiple-choice 
set consisted of one word which was the correct foim class but was semantically 
in^levant and words from two incorrect foiiii classes. The vocabulary used in 
this test was derived from the first forty lessons of the CAE program and the 
Ginn and Alyne and Bacon first-grade readers. 

Pronunciation . The pron\inciation test was composed of forty items, 20 of 
which were real monosyllabic words and 20 items were pronounceable nonsense 
monosyllabic units. Each nonsense item was composed as a minimal contrast to a 
real word item, (e.g., man,fam). Five pairs were constructed for each of four 
categories of minimal contrast: initial consonant, medial vowel, final consonant 

and initial consonant blend. Each item was typed in lower case primary type on 
a 3 X 5 card and presented to the subjects one at a time. The subjects were 
required to attempt a pronunciation of each item. 

Student responses were recorded on magnetic tape for future more detailed 
analysis. The results reported in this paper are those of the all-or-none 
scoring procedure (i.e., an item was scored as correct only if all phonemes of 
the item were correctly pronounced and in proj)er sequence ) , 

Overall Results . The re' alts of each of the above tests and their major 
subsections were examined in a series of three-way analyses of variance (treatment, 
high/low I.Q. , sex). No significant interactions were found in any of the analyses. 
The expected significant differences on the I.Q. and sex variables were found 
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throughout the tests and are of little interest here. The means and standard 
deviations for each test for the experimental and control groups are shovn in 
Table 4. It is interesting to note that with the exception of two cases 
(Stanford Achievement Test, paragraph reading and total score) the direction 
of differences between the means is in favor of the experimental group. 

Insert Table ^ about here 

The test battery may be divided into two major categories: l) tests designed 

to evaluate the goals and linguistic orientation of the CAI program, and 2) tests 
designed to evaluate the outcomes of a quite different approach to initial reading 
(i.e., the traditional basal reading series approach). Differences between the 
means of each of the project developed tests were statisticallj significant and 
in one case, the pronunciation test, the differences were faiily dramatic. Sta- 
tistical significance was also achieved in three of the eight- sub- scales of the 
standardized tests. Two of those sub-scales, however, the vocabulary section 
of the California Achievement Test and the word- study skills section of the 
Stanford Achievement Test, were composed of a series of tests of sub-skills. It 
was necessary, therefore, to investigate the possibility that large differences 
on one or two of the sub- skills might be exerting an influence powerful enough 
to produce significance in the sub- scale as a whole. Accordingly, tlie scores 
on each test of sub-skills were examined in the same three-way analysis of 
variance as described above. The resul : may be seen in Taole 5* The signifi- 
cance levels held in all four of the sub-skills for the ^rord-study section of 
the Stanford Achievement Test and in three cl the five tests of sub-skills in 
the vocabulary section of the Califomua Achievement Test. 

Insert Table 5 about here 
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The Impact of this year's CAT instruction on reading pc forxnance may be more 
readily visualized vben the results are tabulated by reading behaviors which 
were measured in the evaluation program as in Table 6. Eight of the nine tests 
of decoding skills resulted in differences in favor of the CAI group which were 
significant at the .05 level and in six of those eight tests the significance 
was at the .01 level or beyond. Three of the tests of comprehension at less than 
the paragraph level resulted in differences in favor of the CAI group which were 
significant at the .05 level. No significant differences were found in the 
tests of con5)rehension at the paragraph level. 

Insert Table 6 about here 

The above results are most encouraging for the potential impact of CAI on 
initial reading in light of the fact that even the fastest student in this year's 
program progressed only a small fraction of the way through the first year s 
curriculiam (Figure l6). The curriculum was designed with the expectation that 
the able student would be able to complete approximately l80 lessons in the 
first year. As may be seen in Figure l6, the top student in this year's program 
completed ondy approximately 22 percent of the expected total number- of lessons. 
The average student in this year's group completed only 11 percent of the total 
first year's pregram. 

Insert Figure l6 about here 

A comparison of Table 1 and Figure l6 will verify that no student in this 
year's run had been exposed on the CAI system to more than five basic patterns: 
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MEANS AND STANDARD DEVIATIONS: MAJOR TEST SECTIONS 




**Raw Scores 
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MEANS AND STANDARD DEVIATIONS: SUB-SCALES 





Experiment 


Control 


Significance 


Level 




Mean 


S.D. 


Mean 


S.D. 






C.A.T. 

Vocabulary : 
Word Form 


18.3 


5.13 


15.0 


5.30 


p < 0.01 




Word Recognition 


13.h 


4.87 


11-7 


4.50 


0.01 < p < 


0.05 


Opposites 


6.2 


2.96 


5.7 


2.70 


N.S. 




Picture Association 

i 


8.2 


2.92 


6.8 


2.26 


p < 0.01 


\ 


j Letter Recognition 

1 


21.2 


5.17 


20.6 


6.20 


N.S. 


i 


i S.A.T. 

1 

1 Word Study : 
j Audio Perception 

Beginning Sounds 

1 


9.87 


2.71 


8.00 


1 

1 

1 

i 

1 

j 

I 2.70 

t 


p < 0.01 


! 

1 

) 

i 


1 

; Ending Sounds 


8.89 I 

1 


i 2.77 

! 


6.78 


I 3.13 

1 


p < 0.01 


t 

i 

i 


1 

Phonics 

i 


10.20 


2.54 


8.89 


: 2.70 

1 


p < 0.01 


1 

{ 

f 


j Phonograms 

1 


7.28 


1 2.88 

1 


6.33 


2.18 


0.01 < p < 


0.05 
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SIGNIFICANCE LEVEI5 OF TESTS OF READING BEHAVIORS 




Table 6 (cont'd) 



Number of 
Items 



^ Test 

1. C.A.T., Letter Recognition Test 

2. C.A.T., Reading Vocabulary-Word Foim 

Test I, Section A 

3. C.A.T., Reading Vocabulary-Word Recognition, 

Test I, Section B 

4. Hartley Vocabulary Test 

5. S.A.T., Word Study Skills 

Auditory Perception of Beginning Sounds 

6. S.A.T., Word Study Skills 

Auditory Perception of Ending Sounds 

7. S.A.T., Word Study Skills 

Phonics - Recognition of Word from Spoken Cues 

8. S.A.T., Word Study Skills 

Phonograms - Rhjmiing Words 

9. Hartley, Phonetic Discrimination 

10. Hartley, Pronunciation Test 

11. Gates, Vocabulary Test - Picture Association 

12. S.A.T., Word Reading - Picture Association 

13. C.A.T., Vocabulary - Picture Association 

Test I, Section D 

lU. S.A.T., Vocabulary 

15. C.A.T., Vocabulary - Meaning of Opposites, 

Test I, Section C 

16. Hartley: Form Class 

17. Gates, Comprehension Test 

18. C.A.T., Comprehension - Test 2 

19 . S . A . T . , Paragraph Me aning 
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Distribution of Students Over Lessons (1966-67) 



ac, cac, ccac, ic, and cic (where "c" stands for any consonant an3 each vowel 
is specified). This minimal exposure was sufficient to produce a significant 
increase in perfomance in nine out of the ten tests cf decoding skills. Some 
transfer of learning is evidenced by the fact that six of those nine tests were 
commercial standardized tests whose vocabulary and word patterns were not based 
. on the Stanford CAT program. Transfer of training is also born out by the 
observation of the classroom teachers involved, particularly the teacher of the 
low maturity group who has reported that a noticeable percentage of the low group 
of students were discovering word patterns in their classroom reading instruction 
generalizing those word patterns in the attack of new words. 

The most consistent results were found in the area of decoding skills which 
is exactly the area of initial reading which is handled in greatest detail in 
the Stanford CAI program. On the other hand, results in the con5>rehension t‘-sts 
were mixed, with no significant differences being found in tests of paragraph 
con^jrehension. It must be noted that even the fastest stu^.ent in this year’s 
program had not reached the level in the lesson materials where exercises jn 
the comprehension of connected discourse is introduced, xt is also true, as 
stated earlier in this paper, that the comprehension exercises in the Stanford 
CAE curriculum are more of the nature of data gathering devices than they are 
a well defined rnd complex teaching effort. 

FINAL REMARKS 

It has been our goal in this paper to describe the Stanford Reading Pro^iect 
in terms of the CAI system, its operation, its function in the elementary school 
setting, the rationale and major components of the curriculum, and to give some 
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feeling for the processes involved in the production of such a curriculum. We 
have also discussed some results from the first year operation of the program. 

It cannot be emphasized too strongly, however, that the data and the conclusions 
drawn from them are at best tentative; they are based on the first twenty-six 
weeks of operation which must be viewd essentially as an extended debugging 
period for both the system and the learning materials. The progress of the 
students was limited by known problems which are in the process of being remedied 
the next year's operation. Revisions for /iext year involve modification of. 
the branching structure, the display fomiats, and the audio search procedures. 

The projected distribution of students at the end of the school year 1967-68, 
shown in Figure 16, is based upon an experiment run during the last week of 
the 1966-67 operation in which all of the students were placed in selected 
lessons of advanced material where the known problems had been corrected. The 
projection is based not on a pious hope but on some quantity of empirical data. 
The results of this first year's evaluation, while tentative, do suggest that 
such a program undertaken at the end of the 1967-68 school year may demonstrate 
an impact of a CM program on the acquisition of initial reading skills which 
is not only statistically significant but practically significant as well. 

As the curriculum becomes refined and the system achieves a high degree of 
stability and reliability, the Stanford-Brentwood CM laboratory will be made 
available as a laboratory for educational research in the classical sense. It 
will be relatively simple, for example, to design and implement alternative 
teaching strategies which may be evaluated in a highly controlled environment. 
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The problems of optimal teaching strategies may be investigated in a rj.gorous 
manner. It is hoped that basic educational research, vhich may be carried out 
in such a facility, will prove relevant not only to computer-assisted instruc- 
tion but will be generalizable also to the Uiq?ortant problems of teaching and 
learning throughout our educational system. 
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